Frances was the third child in a family of six. Her elder siblings were Esther (Hetty, 1749–1832) and James (1750–1821), her younger Susanna Elizabeth (1755–1800), Charles (1757–1817) and Charlotte Ann (1761–1838). Of her brothers, James became an admiral and sailed with Captain James Cook on his second and third voyages. The younger Charles Burney became a well-known classical scholar, after whom The Burney Collection of Newspapers is named. Her younger sister Susanna married in 1781 Molesworth Phillips, an officer in the Royal Marines who had sailed in Captain Cook's last expedition; she left a journal that gives a principal eye-witness account of the Gordon Riots. Her younger half-sister, Sarah Harriet Burney (1772–1844), also became a novelist, publishing seven works of fiction. Esther Sleepe Burney also bore two other boys, both named Charles, who died in infancy in 1752 and 1754.