Abbott was married to Edna Lewis from 1914 to her death in 1930; they had one child. Actress Mary Sinclair was his second wife. Their marriage lasted from 1946 until their 1951 divorce. He had a long romance with actress Maureen Stapleton from 1968 to 1978. She was 43 and he was 81 when they began their affair, then ten years later Abbott left her for a younger woman. His third wife was Joy Valderrama. They were married from 1983 until his death in 1995.