On 19 July 1822, Percy Jocelyn was caught in a compromising position with a Grenadier Guardsman, John Moverley, in the back room of The White Lion public house, St Albans Place, off The Haymarket, Westminster. He and Moverley were released on bail, provided by the Earl of Roden and others. Jocelyn broke bail and moved to Scotland where he worked as a butler under an assumed name. He was declared deposed in his absence by the Metropolitan Court of Armagh in October 1822 for "the crimes of immorality, incontinence, Sodomitical practices, habits, and propensities, and neglect of his spiritual, judicial, and ministerial duties". A public subscription was raised to raise money for James Byrne, whose 1811 conviction was now recognised as a miscarriage of justice.