By that stage of the war the Union Army occupied most of Arkansas, with several troops under the command of Captain F. S. Dodge enforcing the law in the area of Lafayette County. Most of these Union troops were African American, and despised by Baker. Toward the end of 1864 Baker was in a saloon in the small town of Spanish Bluff, Bowie County,Texas when he was approached by four African American Union soldiers and asked for identification. Baker turned to face them with his pistol drawn, shooting and killing one sergeant and the three other soldiers.