However, he was suspected by MINAMOTO no Yoritomo of agreeing with MINAMOTO no Yoshitsune, and because of pressure from Yoritomo, he again was temporarily removed from the position.
しかし源義経に同意したとされて鎌倉の源頼朝の圧力でまたしても一時的に解官される。