The family descended from James (–1612), a younger son of Robert Elphinstone, 3rd Lord Elphinstone: King James VI and I granted the lands of Balmerino Abbey in Fife to James Elphinstone in 1605, though by the 18th century a series of lawsuits had reduced the family's properties to the barony of Restalrig in South Leith.
The family lineage traces back to James (–1612), a younger son of Robert Elphinstone, 3rd Lord Elphinstone: King James VI and I granted Balmerino Abbey lands to James Elphinstone in 1605. By the 18th century, legal disputes had reduced the family's estates to Restalrig barony in South Leith.