It was while teaching at Washington and Lee University that he helped to found Shenandoah—a publication that featured the debut work of Tom Wolfe among others. He also taught at the University of Brazil and the University of California at Santa Barbara, before assuming the position at USC where he taught for 37 years.
Brown was born and raised in Louisville. Ashley Brown attended Louisville Male High School and went on to complete a master's degree at Yale and a Ph.D. at Vanderbilt.