A brain tumor refers to an abnormal growth of cells in the brain that can be either benign or malignant. Oncologists typically use various methods such as blood or visual tests to detect brain tumors, but these approaches can be time-consuming, require additional human effort, and may not be effective in detecting small tumors. This work proposes an effective approach to brain tumor detection that combines segmentation and feature fusion. Segmentation is performed using the mayfly optimization algorithm with multilevel Kapur’s threshold technique to locate brain tumors in MRI scans. Key features are achieved from tumors employing Histogram of Oriented Gradients (HOG) and ResNet-V2, and a bidirectional long short-term memory (BiLSTM) network is used to classify tumors into three categories: pituitary, glioma, and meningioma. The suggested methodology is trained and tested on two datasets, Figshare and Harvard, achieving high accuracy, precision, recall, F1 score, and area under the curve (AUC). The results of a comparative analysis with existing DL and ML methods demonstrate that the proposed approach offers superior outcomes. This approach has the potential to improve brain tumor detection, particularly for small tumors, but further validation and testing are needed before clinical use.