Rice is a crucial food crop, but it is frequently affected by diseases during its growth process. Some of the most common diseases include rice blast, flax leaf spot, and bacterial blight. These diseases are widespread, highly infectious, and cause significant damage, posing a major challenge to agricultural development. The main problems in rice disease classification are as follows: (1) The images of rice diseases that were collected contain noise and blurred edges, which can hinder the network’s ability to accurately extract features of the diseases. (2) The classification of disease images is a challenging task due to the high intra-class diversity and inter-class similarity of rice leaf diseases. This paper proposes the Candy algorithm, an image enhancement technique that utilizes improved Canny operator filtering (the gravitational edge detection algorithm) to emphasize the edge features of rice images and minimize the noise present in the images. Additionally, a new neural network (ICAI-V4) is designed based on the Inception-V4 backbone structure, with a coordinate attention mechanism added to enhance feature capture and overall model performance. The INCV backbone structure incorporates Inception-iv and Reduction-iv structures, with the addition of involution to enhance the network’s feature extraction capabilities from a channel perspective. This enables the network to better classify similar images of rice diseases. To address the issue of neuron death caused by the ReLU activation function and improve model robustness, Leaky ReLU is utilized. Our experiments, conducted using the 10-fold cross-validation method and 10,241 images, show that ICAI-V4 has an average classification accuracy of 95.57%. These results indicate the method’s strong performance and feasibility for rice disease classification in real-life scenarios.