where is the relative observed agreement among raters, and is the hypothetical probability of chance agreement, using the observed data to calculate the probabilities of each observer randomly seeing each category. If the raters are in complete agreement then formula_3. If there is no agreement among the raters other than what would be expected by chance (as given by ), formula_4. It is possible for the statistic to be negative, which implies that there is no effective agreement between the two raters or the agreement is worse than random.