With considerable experience as a Councillor and Town Clerk, and just five months after his election as mayor of the Corporate Town of Burra, Holder was elected to the South Australian House of Assembly in 1887 as the member for Burra, and soon gained a sound reputation in parliament. As a result, he served as Treasurer of South Australia from 1889–90 in the J. A. Cockburn ministry, and Leader of the Opposition from 1890–92. He again served as Opposition Leader in 1899. He sat on many royal commissions during his parliamentary career in South Australia, and his reasonableness and sincerity made him a very valuable committee man. In June 1891 he carried a vote of want of confidence in the Playford ministry, and took office as Premier and Treasurer. He had only a small majority and it was a time of great financial difficulties due to a severe drought and Holder was forced out as Premier after just four months.