Colman was married to Thelma Raye from 1920 until they divorced in 1934. Then he was married to Benita Hume from 1938 until his death in 1958. They had one daughter. Colman died on 19 May 1958 in Santa Barbara, California from emphysema, aged 67.