It is not exactly unclear when he was born or became a successor of the family; however, because of the description in Rurido Engi (writing about the history) saying 'On April 8, 1169 during the reign of Minamoto no Koreyori', it can be said that at least around then, he had already succeeded to the family because of his father's retirement or death.
惟頼がいつ頃生まれ、いつ頃に家督を継いだかは不明であるが、瑠璃堂縁起という書物に「嘉応元年（1169年）四月八日源惟頼の御世」との記述がある事から、この頃には、父の隠居又は死により、家督を継いでいたものと思われる。