In "Azuma Kagami" (The Mirror of the East), there remains an anecdote that Yoritomo cursed him and other appointees who received the conferment without Yoritomo's previous consent, saying, 'Sueshige HIRAYAMA, he looks woolly-minded, and his appointment is outrageous.'
『吾妻鏡』に他の無断任官者と共に「平山季重、顔はふわふわとして、とんでもない任官である」と頼朝に罵られた記述が残っている。