the town was unusual in that its independently-minded men had a strong control over its clerical life , deliberately limiting the churchs power and influence within the town walls , much to the frustration of the clergy .