Through the efforts of his aunt, Kita no Mandokoro (legal wife of regent or chief adviser to the Emperor), his father's property was protected (25,000 koku stipend of Ashimori in Bicchu Province); however, his fight with his younger brother, Toshifusa, for inheritance of the property resulted in another demotion.
しかし、叔母北政所らの周旋によって父の遺領（備中国足守2万5千石）は安堵されたが、遺領を巡って弟利房と争って再度改易された。