 On 27 July, he first appeared for Olimpia in a friendly match against Fénix which ended 0–0 in Montevideo.He was not taken into account by the club's first-team coach and decided to see if he could play for the club's reserve-team.During a meeting with Olimpia's president, Caballero expressed his desire to play and stated that the year was ending

In January 2006, he was transferred to Club Tacuary for the 2006 Paraguayan Primera División season, Caballero made just two league appearances, however, Tacuary qualified for the 2007 Copa Libertadores as the best finisher in the aggregate points table and the 2007 Copa Sudamericana by winning the 2006 Liguilla Pre-Sudamericana.