When Yoritomo buried the cremains of his father Yoshitomo at Shochoju-in Temple on October 5, 1185, Yoritaka and Yoshinobu HIRAGA carried the palanquin for the cremains and only Yoritaka, Yoshinobu and Koreyoshi OUCHI were allowed to enter the temple, suggesting that Yoritomo continued to take care of Yoritaka very well as a member of the Minamoto clan.
その後も源氏一門として遇され、文治元年（1185年）9月3日(旧暦)、頼朝が父義朝の遺骨を勝長寿院に埋葬した際、遺骨を運ぶ輿を頼隆と平賀義信が運び、頼隆・義信・大内惟義のみを御堂の中に参列させている。