Allen encountered difficulties in Boston. Racial prejudice made it difficult for him to earn a living; in 1845, he wrote a letter to John Jay Jr. (the grandson of the country’s first Chief Justice) discussing the difficulty of finding clients in Boston and wondering whether he would do better in New York City, with its larger African American population. Four years later, Allen was attacked by four men in Boston, although their motivation is unclear. In 1852, Allen’s landlord pressed charges against him for allegedly ripping out parts of his apartment to burn as firewood. A jury acquitted him.