However, in 1992, after less than two years in the Senate, he resigned to return to state politics. The Bannon government was under pressure from the collapse of the State Bank of South Australia. However, Olsen's successor as state Liberal leader, Dale Baker, was unable to gain significant ground. Baker resigned as state Liberal leader in 1992 and called a spill for all leadership positions, intending to hand the leadership back to Olsen as soon as he was securely back in the legislature. To facilitate this, former Deputy Premier Roger Goldsworthy, a leading member of the Liberals' right wing, resigned his seat of Kavel, based on Mount Barker, and handed it to Olsen. However, several members of the party's moderate wing were unwilling to see Olsen take the leadership uncontested. They arranged for leading party moderate Ted Chapman to give up "his" seat of Alexandra and hand it to Brown so Brown could challenge for the leadership. Olsen returned to the House of Assembly at the 1992 Kavel by-election, on the same day as Brown at the 1992 Alexandra by-election. This time, Brown narrowly defeated Olsen in the leadership ballot, and thus became premier when the Liberals won the 1993 election in a landslide where the Liberals won 37 of the 47 seats available, the most that any party has won since the abolition of the Playmander. Olsen became Minister for Industry and Minister for Infrastructure until 1997, when a cabinet reshuffle saw him become Minister for Information Technology and Minister for Multicultural and Ethnic Affairs.