Until 1885, he was a devoted adherent of William Ewart Gladstone, particularly in finance and foreign affairs. In 1880 he was appointed Under-Secretary of State for the Home Department, in 1881 Under-Secretary of State for the Colonies and in 1882 Financial Secretary to the Treasury. He was known as a stubborn fighter for principle, and after finding that the government's Reform Bill in 1884 contained no recognition of the scheme for proportional representation, to which he was deeply committed, he resigned office. He refused to support Gladstone's Home Rule Bill in 1886 and was one of those who chiefly contributed to its rejection, whose reputation for unbending integrity and intellectual eminence gave solidity to the Liberal Unionist party.