In 1831, one of his sons died. Brown became sick, and his businesses started to do poorly. This made Brown have a lot of debt. In the summer of 1832, shortly after the death of a newborn son, his wife Dianthe died. The only family he had left were his children John Jr., Jason, Owen, and Ruth. (Another three of their children died before they became adults.) On June 14, 1833, Brown married 16-year-old Mary Ann Day (April 15, 1817 – May 1, 1884). She was from Washington County, New York. They had 13 children; those alive at John Brown's death were Salmon, Annie, Sarah, and Ellen.