mechanism design problem in which we look to provide incentives for peers to assess well, as well as a means to try to compensate for any biases.

Our main contribution is to propose the PeerRank method for peer assessment. This constructs a grade for an agent based on the grades proposed by the agents evaluating the agent. The PeerRank method makes two basic assumptions about how peer grades should be combined. First, it supposes that the grade of an agent is a measure of their ability to grade correctly. Hence, grades are weighted by the grades of grading agents. Second, agents should be rewarded for grading correctly. This gives agents an incentive to provide accurate peer assessments. We identify some formal properties of the PeerRank method. We also evaluate the performance on some synthetic data. As our method favours consensus, it is most suited to domains where there are objective answers but the number of agents is too large for anything but peer grading.

We hope that this work will encourage others to consider peer assessment from a similar (social choice) perspective. There are other axiomatic properties we could formalise and study. For instance, the PeerRank rule is not monotonic. 