Murray was born on 2 March 1705, at Scone Palace in Perthshire, Scotland, the fourth son of the 5th Viscount of Stormont and his wife, Margaret, née Scott, and one of eleven children. Both his parents were strong supporters of the Jacobite cause, and his older brother James followed "The Old Pretender" into exile. The Jacobite sympathies of Murray's family were glossed over by contemporaries, who claimed that he had been educated at Lichfield Grammar School with many other members of the English judiciary. This was incorrect, as Murray was educated at Perth Grammar School, where he was taught Latin, English grammar, and essay writing skills. He later said that this gave him a great advantage at university, as those students educated in England had been taught Greek and Latin, but not how to write properly in English. While at Perth Grammar School, it became apparent that Murray was particularly intelligent, and in 1718, his father and older brother James decided to send him to Westminster School, as James knew the Dean, Francis Atterbury. The distance from Perth to London was around , and the journey took Murray 54 days. Murray flourished at Westminster and was made a King's Scholar on 21 May 1719.