As diocesan bishop at Hereford Scory had troubles. He wrote to Parker describing the condition of his diocese, which contained many chapels either unserved or served with a reader only; some of the parish churches were in danger, owing to an interpretation of the statute for the suppression of colleges. He also was bothered by the proceedings of the council for the marches of Wales, and had difficulties with the cathedral clergy; but he obtained new statutes for the cathedral in 1582. He was accused of being a money-lender. John Aubrey says that he loved his son Sylvanus Scory "so dearly that he fleeced the Church of Hereford to leave him a good estate". He instituted a thorough review of the lands of the bishopric which was carried out 1577–80 by the young Swithun Butterfield.