Olivier made his provincial rugby debut for the Blue Bulls in the Currie Cup in a match against Boland in 2003. He made his debut in the Super 12 (now Super Rugby) competition in 2005, playing for the Bulls in a game against the Otago-based Highlanders. He was included in Jake White's 2006 Springboks 45 man training squad. Olivier made his test debut in the first test against Scotland in Durban in 2006 and was included in the Bok starting lineup for the tests against Scotland in Port Elizabeth and France in Cape Town. He was selected to tour with the Springboks as they started their Tri Nations quest in 2006.