According to a temple legend, Raigo-in Temple was built by Kukai (Kobo daishi, one of the best known and most beloved Buddhist saints in Japan, founder of the Shingon ("True Word") school of Buddhism) in 806 by enshrining the statue of Kojin (god of a cooking stove) he had happened to obtain in Tang (China).
寺伝によれば、空海（弘法大師）が806年（大同(日本)元年）に唐（中国）で感得した荒神像を安置して来迎院を開創したとされる。