The drive for education instilled by Brown's mother had compelled her to finish high school and advance to the university level. Brown and her younger sister Oella decided to attend Mount Holyoke Female Seminary. Mount Holyoke and a college education were what Brown had hoped for. Her excitement was tempered by the restrictions placed on women at Mount Holyoke. These restrictions included a list of forty rules, the abolition of a literacy society founded by the Browns, and religious restrictions. Perhaps the best example of the school's thinking was the words of a Chemistry professor, "You are not expected to remember all of this, but only enough to make you intelligent in conversation." Brown, who already knew she could meet the challenges of a higher education, looked elsewhere.