Elgie was easily re-elected in the 1981 election, and was named Minister of Consumer and Commercial Relations on February 13, 1982. He supported Roy McMurtry's bid to succeed Davis as party leader in 1985. When Frank Miller replaced Davis as Premier of Ontario on February 8, 1985, he named Elgie as his Minister of Community and Social Services. He was re-elected with a reduced majority in the 1985 election, as the Progressive Conservatives won a narrow minority government under Miller's leadership. Elgie was again appointed as Minister of Labour on May 17, 1985.