Yokoyama-jo Castle was constructed in 1561 on a ridge of the mountain chain on the south side of the Ane-gawa River running between Azai County and Sakata County by Nagamasa AZAI as a base of defense against the Rokkaku clan.
横山城は近江北部の浅井郡と坂田郡とを分ける姉川の南岸の山の峰続きに永禄3年（1561年）、浅井長政が対六角氏用の防衛拠点として築城させた。