In 1040 he doubled as Sakyo no daibu (Master of the Eastern Capital Offices) and he has served as Kurodo no to until he was promoted to Sangi (councilor) in 1042, but during this period his relationship with Yorimichi deteriorated considerably (it is said that such a situation was also deeply related to the fact that he remained in the position of Sangi and he was never promoted to a higher rank).
長久元年（1040年）に左京大夫を兼ねて、長久3年（1042年）の参議昇進まで蔵人頭を務めているが、この間に頼通との関係は大きく悪化した（彼の昇進が参議に留まったのもこのことが深く関係しているとされる）。