Dr. McGhee died on 18 March 1996 in Stirlingshire, of cancer. At his funeral the church was overflowing with over 1500 mourners, and he was buried in Falkirk Cemetery. The Very Revd Sandy McDonald (father of Doctor Who actor David Tennant) preached at his funeral. Dr. McGhee's family donated hundreds of theological books which were part of his private collection to the University of Glasgow. A stained glass window of St. John was erected in his memory in St. Andrew’s West Church, and a new residential street built in 2005 in central Falkirk, "McGhee Place", was named after him.