He enjoys the company of women. From the incident of keeping a girl aboard ship in his youth, unbeknownst to him, she was pregnant when he sailed away. Their son, Samuel Panda, appears in Aubrey's life fully grown and educated, a dark-skinned version of himself, but a Catholic priest. Before he knew of this young man, Aubrey married Sophia Williams, whom he met and courted in the peace of 1802, when he was on land. They married and had three children, twin daughters Fanny and Charlotte, and a son George. He loves his family, though most of the time he is away on a ship.