During the conflict between his uncle Yoshitomo and his father Yoshikata in Kanto who sided with MINAMOTO no Tameyoshi, Yoshitaka was defeated by his nephew Yoshihira; with the help of Shigeyoshi HATAKEYAMA and Sanemori SAITO young Yoshinaka evacuated to Shinano Province (Nagano Prefecture) after his father's defeat, grew up under the wing of Kaneto NAKAHARA, the powerful clan in Kisodani, and went by his popular name of "Jiro KISO."
関東での源為義派の父義賢と、伯父の義朝の対立の過程で、父の義賢が甥の源義平に討たれた後、幼少の義仲は　畠山重能、斎藤実盛らの援助で信濃国（長野県）に逃れ、木曾谷の豪族、中原兼遠の庇護下に育ち、通称を「木曾次郎」と名乗る。