He was not well regarded as Prime Minister and his capabilities were summed up in the rhyme: "Pitt is to Addington as London is to Paddington". As Viscount Sidmouth, he was a very reactionary Home Secretary in the Government of the Earl of Liverpool but he was eventually persuaded to retire in favour of Sir Robert Peel.