Furthermore, not to take action is to let down the moderate Muslim majority". It was reported that members of the public made more than 500 complaints to Scotland Yard about the demonstration.
To refrain from action is to disappoint the moderate Muslim majority.  There were over 500 complaints made by members of the public about the demonstration.