On his father's side, Stafford was descended from Edmund de Stafford, who had been summoned to Parliament as Lord Stafford in 1299. The second Baron had been created Earl of Stafford in 1351. On his mother's side, Stafford was the son of Anne of Gloucester, Countess of Buckingham, daughter of Thomas of Woodstock, Earl of Buckingham (later Duke of Gloucester), youngest son of King Edward III of England. Stafford was an important supporter of the House of Lancaster in the Wars of the Roses, and was killed at the Battle of Northampton in July 1460.