is the sum of these peer graded marks (or equivalently the average of the normalised peer grade). We denote this as the AVERAGE rule.

We studied a number of different distributions of marks amongst the agents (e.g. binomial, normal, uniform). These are discussed in more detail in the next section. We also need a marking model to determine how well the grading agents grade. We used a simple model based on each mark being awarded independently with a probability given by the grade of the grading agent. In our experiments, this means that the agents are effectively answering 10 questions, that the probability of each of these questions being answered correctly is their actual grade, and that the probability of each of these questions being graded correctly is the grade of the grading agent. This gives a distribution of marks that is the sum of two binomials.

For instance, if the actual mark of an agent is 62 out of 100, then we expect their peer grade to be (on average) 6 out of 10. Suppose their work is marked by an agent whose actual mark is 72 out of 100. On the 6 questions that the agent is expected 