He also annoyed many people around him by rebuilding his own residence, Higashi-sanjo-den Palace, so that part of it would look like Seiryo-den Palace of Dairi (Imperial Palace), because young Emperor Ichijo lived there with his mother Senshi, by selecting his sons and brothers such as Michitaka or Michinaga as Court nobles, and by executing unreasonable personnel replacements such as replacing all officials with those who were in his own sect.
また、一条天皇が幼少で母親の詮子とともに兼家の屋敷である東三条殿に居住している事を理由に建物の一部を内裏の清涼殿に模して建て替えたり、道隆や道長ら自分の子弟を公卿に抜擢したり、弁官を全て自派に差し替えるなどの強引な人事を行って顰蹙を買った。