Stevens was married to Susan Tucker Huntington from 1942 until they divorced. Then he was married to Barbara Finch from 1969 until they divorced. He had one child with Huntington and two with French. He lived in Sherman Oaks, Los Angeles, California with his family.