In 1832 he became Member of Parliament (MP) for Aberdeen in the British House of Commons, sitting as a Radical, and remained an MP until his retirement in 1847. Together with his wife, Margaret Gordon the granddaughter of former Governor Walter Patterson, Bannerman returned to the colony of her birth, when he took up the appointment in 1851 as governor of Prince Edward Island. On this occasion he was made a Knight Bachelor. Bannerman instituted responsible government on the island but was removed in 1854 due to political unrest in which he favoured the Reformers. Subsequently, he was governor of the Bahamas until 1857, when he returned north to become governor of Newfoundland, the second governor since responsible government had been granted.