He was survived by a widow, three daughters and two sons, neither of which followed their father into the profession. One became a brewer, the other an architect and one of them (although it is not specified which) was an artist of renown. His daughters on the other hand all died young - one through an accident, the other two in the wreck of the "Princess Alice" when it collided with another boat on the River Thames in September 1878. His wife died in 1899. His brother Edward Flatman (c1807-1884) settled at Chantilly in France and rode four winners of the Prix du Jockey Club.