He was baptized on April 22, 1827 in Trinity Episcopal Church, Pittsburgh, Pennsylvania. He was the youngest of the Foster children, and was named for Stephen Collins, a child friend of the family who died at age 12, and for William Collins, a family friend and well known attorney.