that the observed correlation was sufficiently high to replace peer review by metrics. Others concluded that peer review should be supported by citation analysis. Finally, some concluded that peer review should not be replaced by metrics, even though they found relatively high correlations. This indicates that different researchers draw different conclusions, despite finding similar correlations. One problem is that none of the correlations are assessed against the same yardstick; thus, it is unclear when a correlation should be considered “high” and when it should be considered “low”.

Research Excellence Framework 2014
----------------------------------

The REF 2014 was accompanied by an extensive study into the possibilities of using metrics instead of peer review, known as the *Metric Tide* report [@Wilsdon2015]. This report concluded that citations should only supplement, rather than supplant, peer review. One of the arguments for this conclusion was based on an analysis of how field-normalised citations based on Scopus data correlate with peer review. The report found correlations[^6] in the range of about $0.2$–$0.4$. This is quite low compared with most preceding studies, which found correlations of roughly $0.6$–$0.7$. In contrast to preceding studies, @Wilsdon2015 [Supplementary Report II] analysed the correlation between metrics and peer review at the level 