Chilies undergo multiple stages from field production to reaching consumers, making them susceptible to contamination with foreign materials. Visually similar foreign materials are difficult to detect manually or using color sorting machines, which increases the risk of their presence in the market, potentially affecting consumer health. This paper aims to enhance the detection of visually similar foreign materials in chilies using hyperspectral technology, employing object detection algorithms for fast and accurate identification and localization to ensure food safety. First, the samples were scanned using a hyperspectral camera to obtain hyperspectral image information. Next, a spectral pattern recognition algorithm was used to classify the pixels in the images. Pixels belonging to the same class were assigned the same color, enhancing the visibility of foreign object targets. Finally, an object detection algorithm was employed to recognize the enhanced images and identify the presence of foreign objects. Random forest (RF), support vector machine (SVM), and minimum distance classification algorithms were used to enhance the hyperspectral images of the samples. Among them, RF algorithm showed the best performance, achieving an overall recognition accuracy of up to 86% for randomly selected pixel samples. Subsequently, the enhanced targets were identified using object detection algorithms including R-CNN, Faster R-CNN, and YoloV5. YoloV5 exhibited a recognition rate of over 96% for foreign objects, with the shortest detection time of approximately 12 ms. This study demonstrates that the combination of hyperspectral imaging technology, spectral pattern recognition techniques, and object detection algorithms can accurately and rapidly detect challenging foreign objects in chili peppers, including red stones, red plastics, red fabrics, and red paper. It provides a theoretical reference for online batch detection of chili pepper products, which is of significant importance for enhancing the overall quality of chili pepper products. Furthermore, the detection of foreign objects in similar particulate food items also holds reference value.