On January 11, 1854 in Cass County, Texas Baker married Martha Jane Petty, and for a time he settled. Martha Jane Petty was the daughter of Hubbard and Nancy Petty. However, eight months into his marriage, while out drinking with friends, he became involved in a verbal altercation with a youth named Stallcup. Baker became enraged, grabbed a whip, and beat the boy to near death. There were several witnesses to the incident, and Baker was soon charged with the crime. One of the witnesses, Wesley Bailey, was confronted by Baker at Bailey's home. Baker shot him in both legs with a shotgun, then left him lying in front of his house. Bailey died a few days later. Before he could be arrested for the murder, Baker fled to Arkansas, where he stayed with an uncle. On May 24, 1857, Martha Jane Baker gave birth to a baby girl, Louisa Jane. On June 2, 1860, Martha Jane Baker died. Cullen Baker then returned to Texas, where he left his daughter with his in-laws.