Being a son of MINAMOTO no Mitsunaga who was killed in action during the Jisho-Juei War, Mitsuhira became an adopted son of his uncle MINAMOTO no Mitsumoto and took over the role of family head of the Toki clan.
治承・寿永の乱で討死した源光長の子であるが、伯父源光基の養子となり土岐氏の嫡惣を継承した。