However, he was regarded as being guilty of adultery with FUJIWARA no Shoshi (Taikenmonin and the Empress of Emperor Toba), who was his disciple in regard to the so instrument, in his youth, and as a result, he was unsuccessful as a governmental official.
しかし、若年時に箏の弟子である藤原璋子（待賢門院、鳥羽天皇中宮）と密通したとの嫌疑を受け、その影響から官途の面では不遇であった。