For example, this framework provides several possible measures that are applicable when a researcher wants to assess the consensus between judges who are asked to code a number of items into mutually exclusive qualitative categories. Measures of this latter type have been proposed by several researchers, including Perrault and Leigh in 1989.
This framework provides several possible evaluations that are relevant when a researcher aims to assess the consistency among judges who are tasked with coding a range of items into distinct, non-overlapping categories. Researchers have suggested alternative methods, including those proposed by Perrault and Leigh in 1989.