When his brother Yoshitada returned to the capital, Yoshitoki began to guard Ishikawa of Kawachi Province, the base of the Kawachi-Genji for Yoshitada (the oldest brother MINAMOTO no Yoshimune had died young, the second oldest brother MINAMOTO no Yoshichika had been killed for raising a rebellion, and the third oldest brother MINAMOTO no Yoshikuni had been being confined by the Imperial order for having a battle against his uncle MINAMOTO no Yoshimitsu in the Kanto region.
兄、義忠が都に戻るのと入れ替わりに河内源氏の本拠地、河内国石川を守るようになる（長兄の源義宗は夭折。次兄の源義親は乱を起こし討伐された。三兄の源義国は関東で叔父の源義光と合戦し勅勘を蒙り謹慎）。