In the temple's precincts there are three ancient tombs which were constructed in around the six century, called the Daruma-ji first, second and third tombs, and the hondo (main hall) of the temple is built on the third tomb.
達磨寺の境内には達磨寺1号墳・2号墳・3号墳と称される3基の古墳（6世紀頃の築造）が存在し、このうちの3号墳の上に本堂が建てられている。