After that, it seemed the family's prosperity gradually declined and a person named 'Itsutsuji-nyudo-nomiya' (Imperial Prince and Monk Itsutsuji) visited Imperial Prince Fushiminomiya Sadafusa's place and asked him for a domain in Kyushu in 1432.
その後は暫時衰微したらしく、永享4年（1432年）、伏見宮貞成親王の元に「五辻入道宮」なる人物が訪れ、九州の所領について斡旋を要請した。