In 1526, Yoshitaka constructed Shodoyama-jo Castle in the territory of Mitsugi Betsugu, or in the neighborhood of present-day Miu, Obata-cho, Mihara City, and from that time on, he made it the castle of the Bingo Shibukawa clan.
1526年（大永6年）、御調別宮の領内、現在の三原市小幡町美生のあたりに小童山城を築城し、以後、備後渋川氏の居城とする。