The family descended from James (–1612), a younger son of Robert Elphinstone, 3rd Lord Elphinstone: King James VI and I granted the lands of Balmerino Abbey in Fife to James Elphinstone in 1605, though by the 18th century a series of lawsuits had reduced the family's properties to the barony of Restalrig in South Leith.
The family traced back to James (–1612), a younger son of Robert Elphinstone, 3rd Lord Elphinstone: King James VI and I bestowed the lands of Balmerino Abbey in Fife to James Elphinstone in 1605, but by the 18th century, legal disputes had diminished the family's holdings to the barony of Restalrig in South Leith.