He maintained an intimate relationship with his immediate elder brother Yoshimitsu, and when Yoshimitsu murdered his nephew, MINAMOTO no Yoshitada, in 1109, it is said that he was deeply involved in its planning and execution.
すぐ上の兄の義光と懇意であり、天仁2年（1109年）に義光が甥の源義忠を暗殺した際にはその計画と実行に深く関わったと伝わる。