Maintenance of sufficiently healthy green leaf area after anthesis is key to ensuring an adequate assimilate supply for grain filling. Tightly regulated age-related physiological senescence and various biotic and abiotic stressors drive overall greenness decay dynamics under field conditions. Besides direct effects on green leaf area in terms of leaf damage, stressors often anticipate or accelerate physiological senescence, which may multiply their negative impact on grain filling. Here, we present an image processing methodology that enables the monitoring of chlorosis and necrosis separately for ears and shoots (stems + leaves) based on deep learning models for semantic segmentation and color properties of vegetation. A vegetation segmentation model was trained using semisynthetic training data generated using image composition and generative adversarial neural networks, which greatly reduced the risk of annotation uncertainties and annotation effort. Application of the models to image time series revealed temporal patterns of greenness decay as well as the relative contributions of chlorosis and necrosis. Image-based estimation of greenness decay dynamics was highly correlated with scoring-based estimations (r≈ 0.9). Contrasting patterns were observed for plots with different levels of foliar diseases, particularly septoria tritici blotch. Our results suggest that tracking the chlorotic and necrotic fractions separately may enable (a) a separate quantification of the contribution of biotic stress and physiological senescence on overall green leaf area dynamics and (b) investigation of interactions between biotic stress and physiological senescence. The high-throughput nature of our methodology paves the way to conducting genetic studies of disease resistance and tolerance.