It was originally an annex of a larger temple called Gango-ji Temple, and according to temple legend ASANO no Nakai, the eldest son of the Minister of the Right KIBI no Makibi, received Empress Gensho's former palace and rebuilt it as a temple in the Nara Period.
もとは大寺院だった元興寺の別院とされ、寺伝によると奈良時代に右大臣・吉備真備（きびのまきび）の長男である朝野魚養（あさのなかい）が、元正天皇の旧殿を拝領し創建したと伝わる。