Prevention and management of chronic lung diseases (asthma, lung cancer, etc.) are of great importance. While tests are available for reliable diagnosis, accurate identification of those who will develop severe morbidity/mortality is currently limited. Here, we developed a deep learning model, CXR Lung-Risk, to predict the risk of lung disease mortality from a chest x-ray. The model was trained using 147,497 x-ray images of 40,643 individuals and tested in three independent cohorts comprising 15,976 individuals. We found that CXR Lung-Risk showed a graded association with lung disease mortality after adjustment for risk factors, including age, smoking, and radiologic findings (Hazard ratios up to 11.86 [8.64–16.27]; p < 0.001). Adding CXR Lung-Risk to a multivariable model improved estimates of lung disease mortality in all cohorts. Our results demonstrate that deep learning can identify individuals at risk of lung disease mortality on easily obtainable x-rays, which may improve personalized prevention and treatment strategies.