 He was the oldest son of Charles Coxe of Kemble, Wiltshire and Elizabeth Westley, daughter and coheir of Sir Robert Westley, Lord Mayor of London, 1744–45.He was educated at Christ Church, Oxford.In 1789, he married Anne, daughter of Robert Gordon of Auchendolly, Kirkcudbright, Scotland.