Despite Dolley's weakened position after the death of most of her male relatives, she was still considered a beautiful woman and was living in the temporary capital of the United States, Philadelphia. While her mother went to live with another married daughter in 1801, Dolley caught the eye of James Madison, who then represented Virginia in the U.S. House of Representatives. While remarrying would have been crucial for her, as keeping herself and her child on the income that a woman could earn would have been challenging, it is reported that she did seem to genuinely care for James.