Building work on the Yonghe Temple started in 1694 during the Qing dynasty on the site where originally stood an official residence for court eunuchs of the Ming dynasty. It was then converted into the residence of Yinzhen (Prince Yong), the fourth son of the Kangxi Emperor. The Kangxi Emperor awarded this building to Yinzhen in 1702, who was at the time a "junwang" (second-rank prince). Yinzhen moved into this building in May 1703. The Kangxi Emperor promoted Yinzhen from junwang to qinwang (first-rank prince) under the title "Prince Yong of the First Rank" (和硕雍亲王; 和碩雍親王; Héshuò Yōng Qīnwáng; Manchu: hošoi hūwaliyasun cin wang) in 1709. The name of this building was thus changed into 'the Residence of Prince Yong of the First Rank' (雍親王府) in the same year. In 1711, the fourth son of Yongzheng, or the Qianlong emperor, was born in the East Academy (东书院) in this building.