Biochemical Prostate Cancer Recurrence Prediction: Thinking Fast & Slow
Suhang You Sanyukta Adap Siddhesh Thakur Bhakti Baheti Spyridon Bakas









S.You, et al.



Division of Computational Pathology, Department of Pathology & Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, USA
 Corresponding authors: jaydyou@iu.edu, spbakas@iu.edu 

Biochemical Prostate Cancer Recurrence Prediction: Thinking Fast & Slow





    Suhang You*1 Sanyukta Adap1 Siddhesh Thakur1 Bhakti Baheti1 Spyridon Bakas*1 
    September 2024
=================================================================================



§ ABSTRACT 
Time to biochemical recurrence in prostate cancer is essential for prognostic monitoring of the progression of patients after prostatectomy, which assesses the efficacy of the surgery. In this work, we proposed to leverage multiple instance learning through a two-stage “thinking fast & slow” strategy for the time to recurrence (TTR) prediction. The first (“thinking fast”) stage finds the most relevant WSI area for biochemical recurrence and the second (“thinking slow”) stage leverages higher resolution patches to predict TTR. Our approach reveals a mean C-index (Ci) of 0.733 (θ=0.059) on our internal validation and Ci=0.603 on the LEOPARD challenge validation set. Post hoc attention visualization shows that the most attentive area contributes to the TTR prediction.








§ INTRODUCTION



In 2020, more than 10 million new male cancer cases were diagnosed, with prostate cancer (PC) ranking second to lung cancer <cit.>. Currently, PC clinical treatment relies on prostatectomy targeting prolonged life expectancy. However, up to 40% of PC patients would experience biochemical recurrence of the prostate-specific antigen 
within 10 years <cit.>.
The Gleason score <cit.> has been ranking PC on different risk grades, based on morphological features, albeit its limitations lead to recurrence rate differences within the same grade <cit.>.

 
 

Recently deep learning methods <cit.> have targeted superior biochemical recurrence prediction to the Gleason score, relying on the analysis of digitized histological images of tissue microarrays, rather than whole slide images (WSIs).
A common solution to analyze WSIs is by partitioning them into smaller patches, notwithstanding the challenge of obtaining patch-level annotations. Along these lines, multiple instance learning (MIL) <cit.> has become prominent in computational pathology for many applications <cit.>, as it encapsulates features from individual patches of the same WSI as a bag <cit.>, reducing the patch-level labeling requirement and transforming it into a weakly-supervised learning problem with known bag/WSI-level labels. Direct risk prediction has been proposed in <cit.> by modeling a Cox layer and recently advanced with MIL in <cit.>, which groups the extracted patch-level features with K-means to improve patch sampling.


Motivated by this recent literature, here we propose a two-stage MIL regression approach to tackle the task of predicting biochemical recurrence in prostate cancer, as part of the LEarning biOchemical Prostate cAncer Recurrence from histopathology sliDes (LEOPARD) Challenge 2024 <cit.>. The proposed two-stage approach follows a “thinking fast & slow” strategy, towards improving the patch sampling/pooling and targeting inference efficiency. Specifically, the 1^st stage aims to rapidly localize the most important WSI area, and the 2^nd stage leverages these important patches and focuses on selecting the most attentive features to predict TTR.



§ MATERIAL


We developed our model using the LEOPARD challenge training set (508 cases). We used all training data for our 2^nd stage (Sec. <ref>), and excluded 30% for our 1^st stage (Sec. <ref>) by setting the time threshold T=1.65, where cases with e_i = 0 (no recurrence) and t_i  (follow-up years)  < T are excluded. For both stages, the split ratios for training, validation, and testing were 64%:16%:20%.

UNI was used as the feature extractor, including its pre-trained weights <cit.>.

Our final model was submitted to the LEOPARD Challenge validation and testing phase. The validation set comprised 49 cases from `Radboud' and 50 cases from external sources. The testing set was hidden from challenge participants.



§ METHODS



Our proposed method consists of two MIL-based stages (Fig. <ref>). The 1^st stage (“thinking fast”) targets classification at a low WSI resolution (≈16mpp, ≈0.625Xmagnification), and the 2^nd stage (“thinking slow”) focuses on regression at a high resolution (≈0.25mpp, ≈40Xmagnification). This approach targets improved patch sampling/pooling and inference efficiency. CLAM <cit.> was used for pre-processing (WSI patching and excluding background).


    
    
    
    
    
    

    In the 1^st stage we extracted non-overlapping patches (224×224), whereas patches in the 2^nd stage were of size 2048×2048 with 75% overlap (step size=1024). These were embedded in lower dimensional spaces through UNI and used for classification (recurrence or not) in stage 1, and for TTR regression in stage 2.
    
    

 §.§ Thinking Fast: MIL Classification at Low Resolution

    
    
    
    The 1^st stage intends to facilitate the rapid selection of WSI areas with the largest contribution to the TTR prediction, given a particular time threshold T. Its goal is reduced inference time and increased performance of the proposed approach. The recurrence of the WSI is defined as:
    
    Y|_t = T = 
            0    if ∑_i = 1^Ny_i = 0 
    
            1    otherwise

    where y_i is the prediction for the i^th WSI patch and Y|_t = T is the prediction for the WSI, at time threshold T. For this classification, we apply the CLAM-SB <cit.> model as the “thinking fast” classifier, which includes the patch loss and the cross-entropy for the WSI.
    
    
    
    After prediction, the probability of recurrence for each patch is generated and the top m percent (up to 40%) of patches with the highest attention scores will be assigned 1 in a mask, and 0 otherwise. This mask intends to filter out the less relevant tissue, in preparation for the second stage MIL process. 
    
    
    
    
    
    

 §.§ Thinking Slow: MIL Regression at High Resolution

    Following the work of <cit.>, the “thinking slow” stage is the regression task of predicting the patient risk R for biochemical recurrence. This risk is inversely related to TTR. Thus, the output layer is described by a Cox Proportional Hazard <cit.> (CPH) layer, which is a single node and outputs the logarithmic risk h(S) of a WSI feature embedding S = {f_1,f_2,...,f_N}. The WSI feature embeddings are extracted from patches selected by the mask of the “thinking fast” stage, after being pooled and aggregated for regression.
    
    In the CPH model, the risk R(S) = e^h(S) is estimated by the linear function ĥ_β(S) = β^T · S. In Cox regression, the weights β are optimized by the Cox partial likelihood, which is defined as:
    
    L(β) = ∏_i:e_i = 1e^ĥ_β(S_i)/∑_j:R(t_i) e^ĥ_β(S_j) ,

    where e_i is the event status (recurrence: 1, or not: 0) at follow up t_i (in years), and S_i is the WSI embedding. R(t_i) indicates that the patient, whose input is the WSI, is still at risk of recurrence at time t_i. The optimization of Cox partial likelihood is equivalent to minimizing the following negative log partial likelihood function through re-parameterization:
    
    
    l(β) = - ∑_i:e_i = 1 ( ĥ_β(S_i) - log∑_j:R(t_i) e^ĥ_β(S_j) ),
    
    
    In our design, patch embeddings {f_1,f_2,...,f_N} output their corresponding logarithmic risk {r_1,r_2,...,r_N} through the Cox layer. Then, embeddings with top k logarithmic risk are selected as {f_top_1,f_top_1,...,f_top_k} (Fig. <ref>). Among these embeddings, we define the pooling as a self-attention process <cit.>
    
    
    
    
    
    
    

    
    S ≈ S_top_k = ∑_i = top_1 ^top_k a_i r_i,   
            a_i=exp{𝐰^⊤tanh (𝐕r_i^⊤)}/∑_j=top_1^top_kexp{𝐰^⊤tanh𝐕 r_j^⊤},

    where 𝐰 and 𝐕 are learnable parameters. S_top_k is the top_k embeddings weighted by attention pooling, designed to approximate the WSIs feature embeddings S (Eq. <ref> & <ref>). tanh(·) is an element-wise hyperbolic tangent function, introducing non-linearity. We approximate the TTR using exp(-1 ×log R(S)), since the logarithmic output risk log R(S) is inversely related to TTR.

    
    
    
    
    
    


    

 §.§ Model Training, Evaluation & Selection

    We used the Adam optimizer with a learning rate of 1× 10^-4. The weight decay was 1× 10^-5 and the dropout rate was 0.25. Models were trained and evaluated on NVIDIA A100 GPUs during model selection. Our source code is based on the CLAM platform and the tiffslide library.
    
    To select the best trained “thinking fast” model, we set up a 5-fold cross validation with a fixed test set and select the best fold as the model. The metric is the AUC of prediction on biochemical recurrence. For the “thinking slow” model, we use another 5x5-fold nested cross-validation without a fixed test set. In the outer fold, the hold-out set is used for validation of each inner fold. In each inner fold, the hold-out set is used to select the model for validation on the outer fold hold-out set, where the best inner hold-out validation loss is the criterion during training. The metric we used for model selection is the censored concordance index <cit.> (Ci) of the outer hold-out set. In our setting, 25 Ci are calculated for one parameter setting (e.g., top_k = 10 and m = 20%). In the experiments, we evaluated the model with combinations of top_k = {5, 10, 15, 20, 30, 40, 50} and m = {5%,10%,15%,20%,25%,30%,35%,40%}. We select the model parameters by comparing the best mean and standard deviation (σ) of Ci.
    
    For the model submission to the LEOPARD challenge, we randomly split the data into a 10-fold cross-validation without a testing set and used the best model weights from each fold. The final prediction of TTR is calculated by averaging the predicted logarithmic risk from each set of model weights. We select model weights in each fold, based on the best hold-out validation loss, after 40 epochs. This 40-epoch threshold is set by calculating the zero-crossing epoch of the second derivative of the training loss curve to avoid under-training.
       
    


§ RESULTS




    
    
    
    
    
    
    
    
    
    
    
    
    For the internal data splits (Sec. <ref>), we selected the best performance with parameter settings top_k = 10 and m = 20%. Our proposed approach yielded a mean C-index of 0.733 (σ=0.059) on our test data (i.e., the outer hold-out set), indicating superior performance compared to MAD-MIL <cit.> (0.704 ± 0.058) and AC-MIL <cit.> (0.714 ± 0.056) with regression modifications.
    
    
    Our inference pipeline container submitted in the LEOPARD validation phase, yielded a C-index (Ci) of 0.603 (Ci_Radboud=0.616, Ci_external=0.589).    
    
    
    As shown in Fig. <ref> (A), we compare the results of different combinations of top_k over m percentage values (x axis). The upper plots show mean Ci (y axis) of the outer hold-out set, while the lower plots show their corresponding standard deviation. The best overall result was observed for top_k = 30 and m = 10%. We also observed that using a larger area of the WSI for regression does not always achieve better prediction, which in turn proves that a more relevant area of the WSI provides more accurate features for TTR regression and increases inference efficiency. This phenomenon can also be observed for the other two ablation methods, MAD-MIL and AC-MIL (Fig. <ref>(B)), where the selected method demonstrates a better regression prediction across almost all m parameters when fixing other parameters. 

    
    



§ INTERPRETABILITY
 

In the first stage, the patch selection criterion is the highest attention score (Fig. <ref>) of the attention map, which serves as an interpretability visualization for previous classification works. It show the most attentive area for the stage one classfication. Shown in Fig. <ref>, in our second stage, the attention scores are sparsely distributed on the WSI since only a small portion of patches (10%) are selected. Those color-highlited area also shows the most attentive region for TTR regression. In general, our method leverages the attention mechanism, but further clinical interpretablity requires to be evaluated from clinicians/pathologists.





§ DISCUSSION



In this study, we proposed to leverage MIL through a two-stage “thinking fast & slow” strategy for the TTR regression. The first “thinking fast” stage aims to find the most relevant area of the WSI to the biochemical recurrence and the second “thinking slow” stage leverages higher resolution patches to predict the TTR. In the ablation result, we have shown that an improved prediction can be achieved by focusing on a more relevant area of the WSI along with an improved prediction efficiency. We also showed that the regression is affected by areas of attention which contain cancerous tissues. The limitation of our method is from the CPH model, which focuses on the risk prediction, not the real TTR. In the future, we will extend our work to other tumor types.


§ CODE LINK

 The source code of our inference pipeline is available at <https://github.com/yousuhang/IU-ComPath-LeoPard>.









splncs04

