Gilmore was born on August 25, 1931 in Leipzig, Germany to British parents. He was raised in North Riding of Yorkshire. He was married to Una Stubbs from 1958 until they divorced in 1969. Then he was married to Jan Waters from 1970 until they divorced in 1976. Then he was married to Anne Stallybrass from 1987 until his death in 2013. He had a son with Stubbs. Gilmore died on February 3, 2013 in his home in London, England, aged 81.