 Lowther was born at Sockbridge Hall (his paternal grandmother had been the last of the Lancasters of Sockbridge) and baptised on 4 June 1666.Correspondence suggests he may have been lame.He entered Queen's College, Oxford in 1685, but left after a year, having fallen in with a rakish set there, and taken to drinking and gambling.