Charles Tottenham became Lord of the manor (having to adopt the Loftus name to inherit lands and title as per instructions of Nicolas Loftus 1752) by marrying the Honorable Anne Loftus daughter of the first Viscount Loftus, and they had six children, four boys and two girls – Elizabeth and Anne. However, his wife became ill and died while the girls were still young. Two years later, Tottenham married his cousin Jane Cliffe, and they lived together, along with Anne, in Loftus Hall.