Amberley had two sons and twin daughters, one of whom was stillborn. The eldest child, Frank, was born nine months after their marriage. Rachel Lucretia and her stillborn sister were born prematurely on 2 March 1868, at St. Martin, London, shortly after the couple's return from a tour of North America; the surviving daughter was named after Lucretia Mott, the social reformer whom the couple had met in Boston. In 1870, Lord and Lady Amberley bought Ravenscroft, a country home near Chepstow in Monmouthshire, where he was a magistrate, and spent their time enjoying nature and domestic life. Their youngest and most prominent child, Bertrand, was born in 1872. Amberley asked John Stuart Mill to be Bertrand's secular godfather, and Mill accepted.