Survey and Quality Score Development
comprehension difficulties
MINOR
To ensure feasibility and exclude comprehension difficulties, pretests and pilot tests were conducted between October 26, 2020, and January 19, 2021. The pretests were carried out with 23 pharmacists. The first 12 pretests were executed using the think-aloud method, the remaining using the standard observation before the test. After a revision and adjustment of the survey and rating scheme, a pilot test with 37 pharmacists and advanced pharmacy students as pretesters was performed, resulting in minor modifications. Furthermore, to reduce subjective judgment on each quote’s suitability (item 4d within the rating scheme), 1359 relevant keywords in English and German were identified during the pretest phase and used for a summative content analysis. The frequency of these predefined keywords was measured through the Excel (Microsoft Corp) count function for each quote. To assess whether difficulties occurred during the completion of the search task, participants were asked to rate the clarity of this survey and give qualitative feedback if desired. Data generated by the pretest and pilot test were not included in this study.