Jane Matilda Bolin was born on April 11, 1908 in Poughkeepsie, New York. She was the youngest of four children. Her father, Gaius C. Bolin, was a lawyer and the first black person to graduate from Williams College, and her mother, Matilda Ingram Emery, was an immigrant from the British Isles who died when Bolin was 8 years old. Bolin's father practiced law in Dutchess County for fifty years and was the first black president of the Dutchess County Bar Association.