On April 3, 1882 outlaw Jesse James was killed at his home, originally located at 1318 Lafayette, now sited next to the Patee House. In the post-Civil War years, when the economy was down, the hotel had served for a time as the home of the Patee Female College, followed by the St. Joseph Female College up to 1880. James was living under the alias of Mr. Howard. The song, "Jesse James", includes the lines, "...that dirty little coward that shot Mr. Howard has laid poor Jesse in his grave."