Deep learning feature extraction
DECAY
Before input the images into the deep learning framework, a sonographer who was experienced in kidney ultrasound cropped all the images to contain the entire kidney mask area. The images with the largest ROI of the kidney were then cropped. The cropped kidney mask maintained the complete edge without exceeding the image boundary. Additionally, ultrasound images might be noisy and contain artifacts interfering deep learning process, so we use a traditional wavelet-based denoising algorithm to reduce noise and enhance image features. All the ultrasound images were then resized to a fixed size of 224 × 224 pixels, and the pixel values were normalized to a fixed range of [0, 1] through the z-score method, to improve consistency and reduce complexity. Before the deep learning model training, we performed image augmentation in all the selected images, including rotation (90°, 180°, and 270°), flipping (horizontal and vertical), and color jittering, to meet the need for data diversity in the real world. After image augmentation, additional training data were generated from original images and could improve the robustness of the deep learning model. It also prevented model overfitting and improved the generalization performance of the deep learning model.For our task, we constructed a specific siamese neural network, which consisted of two identical neural networks that shared the same weights. Those two networks are fed with two datasets (US images and SMI images) for the same classification task. ResNet101 has been proven to be a feature extraction tool with good stability and performance [During the transfer learning process, the pre-trained weights of the ResNet-101 model are fine-tuned on altrasound images dataset for our calssification task. Considering the front layers had learned general features that were adapted to many tasks, we freezed the weights of the pre-trained front layers. We trained the last ten layers of the ResNet-101 network so that the later layers could learn more specific and complex features that were more task-dependent. The learning rate was set as 0.001 to control the step size during the gradient descent optimization process. To ensure a relatively faster convergence, the batch size was set as 64 during model training. We also used the regularization methods including dropout and weight decay, which help prevent overfitting of our model. A dropout rate of 0.1 and a weight decay of 0.0001 were used in the ResNet-101 model. The number of epochs in our study was set as 200, which determined the times the model iterated on the entire ultrasound images dataset. We collected the feature vectors in the average-pooling layer of the model as the deep learning features and used the ensemble learning method to calculate the average feature value for each patient. Finally, 2048 SMI-based and 2048 US-based deep learning features were obtained. The image dataset of the testing cohort was input into the trained model to extract deep learning features for further analysis as well.