However, Yorinaga, who, in retrospect, was destined to share his fate with Tameyoshi, was certainly a well-learned, knowledgeable man that no one could have competed with (if there was one, it was FUJIWARA no Shinzei), but he was cold, unloving, inflexible and therefore he met with opposition from people around him, when he insisted on punishing the Taira clan for causing a rebellion by priests from Gionsha Shrine, and he was also hated by Emperor Konoe.
しかし、為義らと運命共同体を成す事となった頼長は確かに学識高く、比類する者（いたとしたら、藤原信西ぐらいであったろう）なきほどであったが、酷薄で融通をつける事を知らず、園社神人と騒動を起こした平氏に厳重な処罰を主張する等周囲から反発を買い、時の近衛天皇にも嫌われていた。