It is said that the temple's history started when the Thousand Armed Kannon Bosatsu (or Deity of Mercy) was enshrined in the same location by EN no Gyoja, who practiced asceticism there.
役行者がこの地に千手観音菩薩を安置し修行したのに始まるという。