Oliver was selected in the second round with the 39th pick by the Cleveland Cavaliers in the 1991 NBA draft. Oliver played his rookie season for the Cavaliers, averaging 3.6 ppg in 27 games. After being invited to the Detroit Pistons' pre-season camp, he joined the CBA for the 1992–93 season, where he averaged 17.3 ppg in 15 games. After one season in the CBA, Oliver returned to the NBA for the 1993–1994 season. He played for the Boston Celtics, where he averaged 4.9 ppg and started six of the 44 games he played. After leaving the league for two seasons, he returned for the expansion draft, where he played a season with the Toronto Raptors for the 1996–97 season. He finished his last two NBA seasons with the Washington Bullets(1997–98) and the Phoenix Suns(1998–99). Playing on five different NBA teams throughout five seasons, he played in a total of 78 games (14 starts) and scored 331 points with 34% three point and .772% free throw accuracy.