His father was defeated in the Heiji Disturbance by TAIRA no Kiyomori in 1159, and Ushiwaka, who was in his 2nd year, ran away in the arms of his mother to the mountains of Yamato Province (Nara Prefecture) with his half brothers, Ano-Zenjo and Gien, who were also born from his mother.
父が平治元年（1159年）の平治の乱で平清盛に敗れ、数え年2歳の牛若は、母の腕に抱かれて2人の同母兄・阿野全成、義円とともに大和国（奈良県）の山中を逃亡した。