His relative obscurity was held against him during the Plot; informers like Stephen Dugdale cunningly invented quite plausible speeches in which he lamented the King's ingratitude and the lack of reward the Howards had received for their loyalty. In fact Stafford, like his fellow Plot victim John Belasyse, 1st Baron Belasyse, thought that under the tolerant regime of Charles II, the Catholic nobility were as well off as they could reasonably expect to be; at his trial he maintained that he had always argued that "we (i.e the Catholic peers) have no other interest than to be quiet." For example, it was well known to the authorities that the Catholic Mass was regularly celebrated at his London town house, but no action was taken against him as a result.