represented about $20\%$ of the total budget of HEFCE of £$3\,602$M and about $40\%$ of the total research budget of HEFCE of £$1\,606$M. As such, it is a sizeable proportion of the total budget.

Literature Review
=================

We review previous literature on how metrics compare to peer review in previous research assessment exercises in the UK. We then briefly review literature that analyses how metrics and peer review compared in the REF 2014.

Research Assessment Exercise
----------------------------

In 1986, the University Grants Committee (UGC) undertook the first nationwide assessment of universities in the UK, called the research selectivity exercise. Its primary objective was to establish a more transparent way of allocating funding, especially in the face of budget cuts [@Jump2014]. Only two years later, @Crewe1988 undertook the first bibliometric comparison of the results for Politics departments in the first 1986 exercise. The results of the 1986 exercise were announced per cost centre (resembling somewhat a discipline or field) of a university in terms of four categories: outstanding, above average, about average, and below average. This limited the possibilities for bibliometric analysis somewhat, and @Crewe1988 only made some basic comparisons based on the number of publications. He concluded that “there is 