These complex arrangements, sealed with mutual judicial and religious oaths, clearly demonstrate that Harun was conscious of their precariousness, in view of the profound differences between al-Amin and al-Ma'mun, both in character and in interests. Very quickly, this latent rivalry had important repercussions: almost immediately after the court returned to Baghdad in January 803, the Abbasid elites were shaken by the abrupt fall of the Barmakid family from power. On the one hand, this event may reflect the fact that the Barmakids had become indeed too powerful for the Caliph's liking, but its timing suggests that it was tied to the succession issue as well: with al-Amin siding with the "abnāʾ" and al-Ma'mun with the Barmakids, and the two camps becoming more estranged every day, if al-Amin was to have a chance to succeed, the power of the Barmakids had to be broken. Indeed, the years after the fall of the Barmakids saw an increasing centralization of the administration and the concomitant rise of the influence of the "abnāʾ", many of whom were now dispatched to take up positions as provincial governors and bring these provinces under closer control from Baghdad. This led to unrest in the provinces, especially Khurasan, where local elites had a long-standing rivalry with the "abnāʾ" and their tendency to control of the province (and its revenues) from Iraq. The harsh taxation imposed by a prominent member of the "abnāʾ", Ali ibn Isa ibn Mahan, even led to a revolt under Rafi ibn al-Layth, which eventually forced Harun himself, accompanied by al-Ma'mun and the powerful chamberlain ("hajib") and chief minister al-Fadl ibn al-Rabi, to travel to the province in 808. Al-Ma'mun was sent ahead with part of the army to Marv, while Harun stayed at Tus, where he died on 24 March 809.