In 1896, Ponsonby became the private secretary to the Governor of the British Cape Colony. Over the next twenty years, he was private secretary to the Governor of Natal (1896), to the Governor of Trinidad and Tobago (1898), to the Governor of Ceylon (1900), and to the Governor of Hong Kong (1903). He was re-posted to assist the Governor of Natal in 1907 and the Governor of Fiji in 1910.