Conclusions
bipolar
In this randomized trial, the use of a rating or ranking procedure led to a modestly similar prioritization of items in a Delphi survey with 30 panelists. We found that both experimental groups identified the same highest priorities but also that discrepancies became increasingly frequent when moving away from the most favored items. The ranking task was perceived as significantly more difficult. Time to complete the study questionnaires showed no statistically significant difference. The embedded trial design was restricted by the requirements of the main Delphi study. Our study should be replicated with a larger number of participants and with variations in the ranking and rating procedures (e.g., ranking within larger lists and rating with smaller, larger or bipolar scales). When determining how to assess items in future Delphi surveys, investigators may favor rating unless they specifically seek to avoid potential ceiling effects.