In 1185, he obtained a court rank without Yoritomo's permission, and was reviled by Yoritomo as 'eyes that are meek like a mouse's and can only follow people, how could he get a position?' ("Azuma Kagami" (The Mirror of the East), entry for May 23, 1185)
同年、頼朝の許しを得ずに官位を得た事で、頼朝に「目は鼠眼にて、ただ候ふべきのところ、任官希有なり」と罵倒されている（『吾妻鏡』1185年（元暦2）4月15日条）。