He would get complaints from the white population saying he was instigating aggravators for promoting the movement; while there were complaints from the movement saying that he wasn't promoting the movement enough. Based on this he couldn't satisfy both sides of the argument in the nation.
The white population complained about his actions that were seen as aggravating issues and advancing the movement; meanwhile, members of the movement felt he wasn't doing enough to promote it. As a result, he couldn't address both groups adequately within the nation.