In September 1863, because Geigi Kotora of Yoshida-ya Geisha House, with whom SERIZAWA was in love, refused his advances he became angry and marched into Yoshida-ya, terrified the owner by saying that he would destroy the Geisha House; he also summoned Kotora and her attendant, Geigi Oshika, and committed the outrage of making them cut their hair ("Roshi Bunkyu Hokoku Kiji").
文久3年9月、芹沢が懸想していた吉田屋の芸妓小寅が肌を許さなかったため、立腹した芹沢が吉田屋に乗り込み、店を破壊すると主人を脅して、小寅と付き添いの芸妓お鹿を呼びつけ罰として2人を断髪させる乱暴を行っている（「浪士文久報国記事」）。