Gabor filter bank has been successfully used for false positive reduction problem and the discrimination of benign and malignant masses in breast cancer detection. However, a generic Gabor filter bank is not adapted to multi-orientation and multi-scale texture micro-patterns present in the regions of interest (ROIs) of mammograms. There are two main optimization concerns: how many filters should be in a Gabor filter band and what should be their parameters. Addressing these issues, this work focuses on finding optimizing Gabor filter banks based on an incremental clustering algorithm and Particle Swarm Optimization (PSO). We employ an SVM with Gaussian kernel as a fitness function for PSO. The effect of optimized Gabor filter bank was evaluated on 1024 ROIs extracted from a Digital Database for Screening Mammography (DDSM) using four performance measures (i.e., accuracy, area under ROC curve, sensitivity and specificity) for the above mentioned mass classification problems. The results show that the proposed method enhances the performance and reduces the computational cost. Moreover, the Wilcoxon signed rank test over the significance level of 0.05 reveals that the performance difference between the optimized Gabor filter bank and non-optimized Gabor filter bank is statistically significant. 
Optimized Gabor features for mass classification in mammography