Montgomery spent two years with the Hampton band. Fear kept him from flying with the rest of the band, so he drove from city to city, town to town, while musicians marveled at his stamina. When arriving at a club, the first thing he did was call home to his wife and family. He was given the opportunity to play with Charles Mingus, Milt Buckner, and Fats Navarro, but not the opportunity he hoped for, and he returned to Indianapolis a better player, though tired and discouraged. He resumed performing at local clubs, this time with the Eddie Higgins Trio and the Roger Jones Quintet, playing with Eddie Higgins, Walter Perkins, and Leroy Vinnegar. He joined his brothers Buddy and Monk and saxophonist Alonzo "Pookie" Johnson in the Johnson/Montgomery Quintet, somewhat in the style of George Shearing. The band auditioned for Arthur Godfrey and recorded sessions with Quincy Jones. After a residency at a club from 1955 to 1957, Montgomery and his brothers went west.