Rating and ranking procedures
During the second and third rounds, panelists were sent the questionnaire version that made use of their assigned assessment mode to appraise the items. Aside from specific instructions regarding each assessment procedure and related survey design questions, the content of both versions of the survey questionnaires was identical. In the second-round questionnaires, items were assessed within their thematic list on separate pages. To prevent order effects, the lists and items within each list were presented in random order to each panelist. Rating panelists used a 7-point unipolar scale ranging from 1 “not at all” to 7 “extremely” important. Numeric and verbal labels were added for each response category to reduce measurement error and increase reliability and validity [During the final round, panelists received structured statistical and qualitative feedback from their study group including, for each item: the person’s score, the group’s median rating or ranking, the interquartile range (IQR), and a summary of positive and negative reasons formulated. They were asked to reappraise only a subset of them to keep their focus on potential top priorities. This subset was produced independently in the two study arms. In the rating group, it included items scored 6 or 7 by at least two-thirds of panelists. In the ranking group, it included items ranked in the top half of their list by at least two-thirds of panelists. These criteria seemed sufficiently comparable as the items had been freely elicited as important from the outset, and we anticipated that most ratings would lie between 4 and 7. This time around, the items were presented on a single list in descending order of importance, as recommended by Delphi methodologists [