Tomato leaf diseases have a significant impact on tomato cultivation modernization. Object detection is an important technique for disease prevention since it may collect reliable disease information. Tomato leaf diseases occur in a variety of environments, which can lead to intraclass variability and interclass similarity in the disease. Tomato plants are commonly planted in soil. When a disease occurs near the leaf’s edge, the soil backdrop in the image tends to interfere with the infected region. These problems can make tomato detection challenging. In this paper, we propose a precise image-based tomato leaf disease detection approach using PLPNet. First, a perceptual adaptive convolution module is proposed. It can effectively extract the disease’s defining characteristics. Second, a location reinforcement attention mechanism is proposed at the neck of the network. It suppresses the interference of the soil backdrop and prevents extraneous information from accessing the network’s feature fusion phase. Then, a proximity feature aggregation network with switchable atrous convolution and deconvolution is proposed by combining the mechanisms of secondary observation and feature consistency. The network solves the problem of disease interclass similarities. Finally, the experimental results show that PLPNet achieved 94.5% mean average precision with 50% thresholds (mAP50), 54.4% average recall (AR), and 25.45 frames per second (FPS) on a self-built dataset. The model is more accurate and specific for the detection of tomato leaf diseases than other popular detectors. Our proposed method may effectively improve conventional tomato leaf disease detection and provide modern tomato cultivation management with reference experience.