Travers was born in Prudhoe, Northumberland, and was the son of Daniel Heagerty, a doctor originally from Ireland, and Ellen Gillman Hornibrook Belcher. His mother was a native of County Cork, Ireland, and was previously married to William H. Belcher, a merchant seaman. He died in 1869. Travers had a half-brother, Samuel William Belcher, by his mother's previous marriage. He also had another brother, Daniel George Belsaigne Heagerty, and a sister, Mary Sophia Maude Heagerty. Travers grew up in Berwick-upon-Tweed, and many biographies wrongly report him as being born there.