He was briefly married to Katharine Foley from 1929 until they divorced months later. Then he was married to Joan Fontaine from 1946 until they divorced in 1951. Then he was married to Ann Rutherford from 1953 until his death in 1991. 