On Mt. Kinugasa, where Kannonsho-ji Temple is located, there was Kannonji-jo Castle, which was the castle of Sasaki-Rokkaku clan who had ruled the southern half of Omi Province since the Muromachi period, and the temple prospered under the patronage of the Sasaki-Rokkaku clan.
観音正寺が位置する繖山には、室町時代以来近江国南半部を支配した佐々木六角氏の居城である観音寺城があり、寺は佐々木六角氏の庇護を得て栄えた。