Cholmeley lost some favour with the City of London authorities because of his reaction to the Evil May Day riots of 1517. During the riots, he furiously ordered the firing of some of the Tower's artillery at the city during rioting by gangs of young Londoners, who attacked foreigners, especially the wealthy foreign merchants and bankers of Lombard Street, London and who took control of London for several days. This drew the ire of the city elders. In 1520, he resigned his post at the Tower due to ill health. He died in March 1521 (1522 by the modern calendar system) in St Katharine's by the Tower.