Both Emperor Xiaowu and Sima Daozi were described to be spending much of their times with drinking and feasting, rather than with important affairs of state. A number of politicians who wanted power gathered around Sima Daozi and flattered him, causing him to be less and less deferential to the emperor. He was also known for being obsessed with gathering wealth and for extravagance in living—including constructing an artificial hill in the back of his mansion, an excessively expensive and difficult task. Emperor Xiaowu eventually became angry about the situation, and he commissioned the well-known officials Wang Gong (王恭) and Yin Zhongkan (殷仲堪) to be provincial governors to counteract Sima Daozi's authority. The emperor and the prince at times suspected of each other, requiring intervention of their mother Empress Dowager Li. However, eventually they reconciled. It was around this time that Sima Daozi would, fatefully, inadvertently offend Huan Wen's son Huan Xuan the Duke of Nan Commandery—as at one feast where Sima Daozi invited Huan Xuan as a guest, after Sima Daozi became drunk, he made the statement, "Was it not true that when Huan Wen became old, he planned treason?" Huan Xuan was so struck by the statement that he fell prostrate on the ground, fearing that Sima Daozi would kill him, and from this point he bore a grudge against the prince.