Yu Di was not happy serving in this manner. In or shortly before 813, there was a man named Liang Zhengyan () who claimed to be a close relative of the powerful eunuch Liang Shouqian (), and Yu had his son Yu Min () bribe Liang Zhengyan, seeking to be made a military governor again. However, at a later point, Liang Zhengyan was revealed to be a fraud, and Yu Min demanded the return of the bribe, which Liang Zhengyan refused. In anger, Yu Min trapped Liang Zhengyan's servant by trick, and killed and dismembered the servant. When this was discovered, Yu Di and Yu Jiyou went to the palace gate in mourning clothes to seek mercy, but the palace guards refused to let him meet the emperor. He was subsequently demoted to be the assistant to Emperor Xianzong's granduncle Li Lian () the Prince of En, and he was denied further access to the emperor's presence. Yu Min was exiled to Lei Prefecture (, in modern Zhanjiang, Guangdong), while Yu Di's other sons were all demoted in rank. On the way to exile, Yu Min was ordered to commit suicide, while Liang Zhengyan and his associate, the Buddhist monk Jianxu () were executed. Later that year, Yu Di was elevated to the higher position of "Taizi Binke" (), an advisor to Emperor Xianzong's crown prince Li Heng.