He left his post of Director General in 1904, giving way to younger generations in accordance with his belief that what was most harmful to business progress was not mistakes of the young but infestation of the old (however, he continued to serve as an adviser to Sumitomo's businesses at request of Tomoito SUMITOMO, the head of the Sumitomo family at the time).
1904年（明治37年）に「事業の進歩発展に最も害するものは、青年の過失ではなくして、老人の跋扈である」として総理事を辞し引退（ただし家長の友純の説得により住友家関係の顧問等は続けた）。