However, he collided with opposing forces within Usa-jingu Shrine, and he was dismissed between January and February of 1005 from Dazai no gon no sochi due to their accusations.
しかし、宇佐神宮内の反惟仲派との衝突で訴えられたのが元で、寛弘元年（1004年）の12月に大宰権帥を解任される。