Local gossip had it that he was not actually married to the woman he was living with, or that there was something amiss in her past. She apparently had been on intimate terms with Mrs Hobson and others in the Government House circle but after William Hobson's death in 1842 and Mrs Hobson's return to Britain she found herself shut out of genteel society in Auckland. This pained Eleanor Freeman enormously and enraged her husband. Living in a workers cottage on the wrong side of town certainly couldn't have helped her social status. After her early death at the age of 26 on 17 December 1844 Freeman added to the wagging tongues by remarrying just six weeks later. After his marriage to Mary Ann Miller on 29 January 1845 the Freemans left New Zealand.