According to the Heian period "Engishiki" a Shinto shrine, the Haritsuna Shrine was moved to make way for the castle. The structure was rebuilt several times in the Muromachi period and the current configuration was largely the work of Oda Nobukatsu, Oda Nobunaga's son. The antiquated architectural style of the watchtower atop the "tenshu" has in the past led many historians to believe this to be the oldest extant "tenshu" in Japan, which was confirmed through tree rings in the construction materials dating the structure to the 1580s. Construction and renovations continued through 1620.