Bannister aspired to work as a painter, but had difficulty finding an apprenticeship or academic programs that would accept him, likely due to racial prejudice. At the time, Boston was an abolitionist stronghold, but it was also sharply divided by race; in 1860, Boston was one of the most segregated cities in the US. Bannister would later express his frustration with being blocked out of artistic education: "Whatever may be my success as an artist is due more to inherited potential than to instruction" and "All I would do I cannot ... simply for the want of proper training."