It is said that the grieving Ningai created a Ryokai Mandala (Mandala of the Two Realms) from the hide of the ox which was installed as the principal image and that this is the reason why the temple was named 'Gyuhisan Mandara-ji Temple' (Ox-hide Mandala Temple).
それを悲しみその牛の皮に両界曼荼羅を描き本尊としたことに因んで、「牛皮山曼荼羅寺」と名付けたという。