Susan Ferrier was the youngest daughter of Helen Coutts (1741–1797) (daughter of Robert Coutts, a farmer near Montrose) and James Ferrier (1744–1829), Writer to the Signet and one of the principal clerks of the Court of Session, in which office he was a colleague of Sir Walter Scott. Her father came from Linlithgow. She was probably born at Lady Stair's Close, Edinburgh, as the ninth of ten surviving children. The family moved in 1784 to 11 (now 25) George Street in the New Town.