In 1949, Fischer-Dieskau married the cellist Irmgard Poppen. Together they had three sons: Mathias, who became a stage designer,Martin (a conductor), and Manuel (a cellist). Irmgard died in 1963 of complications following childbirth. Afterwards, Fischer-Dieskau was married to the actress Ruth Leuwerik, from 1965 to 1967, and Kristina Pugell, from 1968 to 1975. From 1977 until his death he was married to the soprano Julia Varady.