We aimed to clarify the differences in causes of death among patients with breast cancer according to age at diagnosis and years elapsed since diagnosis. Using data from the Osaka Cancer Registry and Vital Statistics databases, 40,690 female patients diagnosed with primary breast cancer between 1985 and 2006 were included in this study. The statistics on all deaths between 1985 to 2016 were collected, and the observation period was 10 years (2006–2016). Mortality hazards according to age at diagnosis and years elapsed since diagnosis were estimated using a flexible parametric estimation. Of the 40,690 patients, 13,676 (34%) died from all-cause death, and the 10-year survival rate was 65.74% (95% confidence interval: 65.28–66.21). The proportions of deaths were 10,531 (77%) from breast cancer, 1048 (8%) from other cancers, and 2097 (15%) from non-cancer causes. The mortality hazard for deaths from breast cancer was initially high and then declined, whereas that for deaths from other cancers and non-cancer causes was initially low and then increased. The more likely causes of death 5 years after breast cancer diagnosis were other cancers or non-cancer causes among patients aged ≥70 years.