He had a fief at Watanabe, Settsu Province (today's Chuo Ward, Osaka City); Yorimasa took the role of Ouchi Shugo, which is equivalent to the guardsman of the Imperial Court, and used as retainers the samurai of the Watanabe clan of the Saga-Genji (Minamoto clan), who served as Takiguchi no musha, or samurai guards of the Imperial Residence.
摂津国渡辺（大阪市中央区(大阪市)）を基盤とし、当地の滝口武者の一族である嵯峨源氏の渡辺氏を郎党にして大内守護（皇室警護の近衛兵のようなもの）の任に就いていた。