Butler left the ministry in controversial circumstances. The report of the Royal Commission on the Wheat Scheme appeared to reflect on the actions of Butler while he was the minister in charge of it, and Peake asked Butler to resign. He refused to do so because he considered that that would admit the justice of the charges. The Executive Council, on the advice of the government, thereupon dismissed Butler from his offices. The report of another royal commission presented some 14 months later was, however, accepted as clearing him of guilt; also the fact that he was elected Speaker of the South Australian House of Assembly in 1921 suggests there had been injustice. Butler was defeated in his seat at the 1924 election after having represented the same district area for 34 years.