Holder was born on August 1, 1930 in Port of Spain, Trinidad, Trinidad and Tobago. He studied at Queens Royal College. He was married to Carmen de Lavallade from 1955 until his death in 2014. Holder had a brother, Boscoe Holder. 