Chase did not surface in public records again until three years later, when she was in the company of Charlie Mayes, also known as Pighead Hardman. On February 15, 1926, Chase, Mayes, and Lee Flournoy and his wife were arrested after a free-for-all fight in a rooming house in Wichita, Kansas. During the arrest, Chase refused to talk. The investigation led investigators to her brother-in-law Charles Chase and allegations of involvement with the Joe Bratton liquor gang. On June 9, 1926, following a "drunken party and joy ride", Flournoy and Mayes were fatally shot in a gun battle in Picher, Oklahoma during which Chase was present. The three of them had been under surveillance by Ottawa County, Oklahoma officers for several days because the deputy sheriff had informed the Sheriff of Montgomery County, Kansas that he had found the suspects who robbed the Cherryvale Bank on May 29, 1926. Chase was placed in jail, where she refused to face reporters. She was released on June 13, 1926, after insufficient evidence was found to charge her with a crime.