Increasing the grade for an agent can hurt an agent if they thereby receive a bigger proportion of their support from agents that grade poorly. On the other hand, the PeerRank rule likely satisfies a more complex form of monotonicity, in which reducing the error in the grade of an agent only ever helps that agent. We expect there are important axiomatic results to be obtained about peer assessment. Finally, an interesting extension would be to return a distribution or interval of grades, reflecting the uncertainty in the estimate.

PEER RANK RULE
==============

We suppose there are $m$ agents, and agent $j$ provides a grade $A_{i,j}$ for the exam of agent $i$. Grades are normalised into the interval $[0,1]$. We suppose agents grade their own work but this can be relaxed. In addition, as we show in the experimental section, the proposed PeerRank rule is relatively insensitive to any bias that an agent might have towards grading their own work or that of other agents. The grade of each agent is constructed from the grades of the agents evaluating the agent. Since the grade is a measure of their ability to grade correctly, we weight the grade an agent gives 