Though only just announced, the exchange happened ahead of a historic meeting between the two sides' presidents on 7 November.
Beijing freed Chu Kung-hsun and Hsu Chang-kuo, who were held in China for nine years, while Taiwan released Li Zhihao, who was jailed 16 years ago.
Taiwanese media say it is the first time the two sides have swapped spies.
Taiwanese presidential spokesman Charles Chen said in a statement that the release was "based on a mutual goodwill gesture delivered by the Ma-Xi meeting".
"President Ma (Ying-jeou) hopes cross-strait mutual exchanges can continue and make more concrete achievements in the future," he added.
China's Taiwan Affairs Office confirmed the release of Col Chu Kung-hsun and Col Hsu Chang-kuo, saying they were freed "in accordance with the law".
Meanwhile, Taiwan said it gave advance parole to Li Zhihao, who had served part of his term.
Defence officials say the two colonels were the last Taiwanese military officials held in China for spying, but that some Taiwanese civilians convicted of spying remained imprisoned in China, the BBC's Cindy Sui reports from Taipei.
President Ma has been broadly friendly towards mainland China during his time in office, boosting tourism and trade ties.
Earlier this month, he and Chinese President Xi Jinping met in Singapore - the first time the leaders of China and Taiwan had held talks in more than 60 years.
China views Taiwan as a breakaway province which will one day be reunited with the mainland.
But many Taiwanese see it as independent and are concerned at China's growing influence.
Taiwan and China swapped jailed spies earlier this month, in a mutual gesture of goodwill.