Although legally the daughter of Sir Henry Hozier and Lady Blanche Hozier (a daughter of David Ogilvy, 10th Earl of Airlie), her paternity is a subject of much debate, as Lady Blanche was well known for infidelity. After Sir Henry found Lady Blanche with a lover in 1891, she managed to avert her husband's suit for divorce because of his own infidelities, and thereafter the couple separated. Lady Blanche maintained that Clementine's biological father was Capt. William George "Bay" Middleton, a noted horseman; Mary Soames, Clementine's youngest child, believed this. However, Clementine's biographer, Joan Hardwick, has surmised (due in part to Sir Henry Hozier's reputed sterility) that all Lady Blanche's "Hozier" children were actually fathered by her sister's husband, Algernon Bertram Freeman-Mitford, 1st Baron Redesdale (1837–1916), better known as a grandfather of the famous Mitford sisters of the 1920s. Whatever her true paternity, Clementine is recorded as being the daughter of Lady Blanche and Sir Henry.