The reason is: Imperial Prince Tamehira's wife was a daughter of Sadaijin MINAMOTO no Takaakira, and Saneyori and Moritada did not want Takaakira of the Minamoto clan to be a maternal relative in the future.
これは為平親王の妃が左大臣源高明の娘であり、実頼と師尹が源氏の高明が将来外戚となることを恐れたためであった。