Bayley's paternal grandfather, Dr. Richard Bayley, was a professor at Columbia College who created New York's quarantine system. Dr. Bayley had three children by his first wife, among whom was Elizabeth Ann Seton, who was canonized in 1975 as the first American-born Roman Catholic saint. After his first wife's death, Dr. Bayley married Charlotte Amelia Barclay, a member of the Roosevelt family, and the couple had seven children, the sixth of whom was Archbishop Bayley's father, Guy Carleton Bayley, born in 1786. Guy Carleton Bayley, a physician like his father, married his second cousin Grace Roosevelt, six years his junior, on November 4, 1813. Grace Roosevelt was the daughter of Jacobus Roosevelt and Maria Eliza Walton, and her brother, Isaac Roosevelt, was the grandfather of future President Franklin D. Roosevelt, making Archbishop Bayley a first cousin to President Roosevelt's father James Roosevelt I, and a fourth cousin twice removed of future President Theodore Roosevelt.