After the Chinese Civil War in 1949, there were many Chinese Muslims fled Mainland China to Taiwan with the Nationalist (Kuomintang) government army. The first mosque was not enough to accommodate all of them, therefore they had to find a new place to rebuild the mosque. The new mosque was proposed by CMA Director-General Bai Chongxi and ROC Minister of Foreign Affairs George Yeh. The mosque was finally built at the Xinsheng South Street in Taipei. The mosque was inaugurated on 13 April 1960 by ROC Vice President Chen Cheng.