Wise was born in the Sydney suburb of Petersham. He was the second son of Edward Wise, a judge of the Supreme Court of New South Wales, and Maria Bate (née Smith). After his father's death in 1865, his mother took the family to Leeds, England to put her sons through grammar school, where their "homemade clothes exposed us to ridicule and bullying". She moved to Rugby and took work, so that Wise could be educated at Rugby School as a day student.