Evidently, the citizenry mistrusted the two lovers' influence over the young duke, for whose safety they may have feared.
The citizens evidently had concerns about the two lovers' impact on the young duke; they might have been worried about their influence on his safety.