He was for the third time returned for the Elgin burghs at the general election in May 1754, but vacated his seat on his appointment as an ordinary lord of session and a Lord of Justiciary in the place of Patrick Grant, Lord Elchies. He took his seat on the bench on 14 November 1754, and assumed the title of Lord Prestongrange. In the following year he was appointed one of the commissioners for the annexed estates. Grant died at Bath on 23 May 1764, aged 63, and was buried on 7 June following in the aisle of Prestonpans Church, Haddingtonshire, where a monument in the churchyard was erected to his memory.