 He was the son of banker Byam Kerby Stevens (1792–1870) and Frances (née Gallatin) Stevens (1803–1877).His father inherited Stevens House, the historic home of his grandfather, Maj. Gen. Ebenezer Stevens.Among his siblings was Albert Gallatin Stevens, Frances Mary Stevens (wife of Rev.