Eldon was born in Newcastle upon Tyne. His grandfather, William Scott of Sandgate, a street adjacent to the Newcastle quayside, was clerk to a fitter, a sort of water-carrier and broker of coals. His father, whose name also was William, began life as an apprentice to a fitter, in which service he obtained the freedom of Newcastle, becoming a member of the guild of Hostmen (coal-fitters); later in life he became a principal in the business, and attained a respectable position as a merchant in Newcastle, accumulating property worth nearly £20,000.