Masur has been married three times. His first wife, with whom he had a daughter, died in 1972 in a car accident. Masur himself was badly injured in that accident. Until his death, he lived with his third wife, Tomoko. They have a son, Ken-David, who has also become a conductor.