In 1955-56, Ray was overlooked and spent four years out of first-class cricket, and he never played alongside Neil again at first-class level. Neil played in all of Victoria's matches, and Mick played in Queensland's first six matches before being dropped for the second and final match against Victoria. In 1956-57, Mick played in two matches for Queensland earlier in the season before being dropped and retiring; neither were against Victoria, so none of the brothers played together in one match. Ray was recalled to the Victorian team in the latter half of the 1958-59 season. By this time, Neil had moved to New South Wales for employment reasons, and played each other late in the season. The Harvey brothers never played together or against one another again at first-class level. In 1959-60, Neil was in the Indian subcontinent representing Australia as Ray played his final first-class season before retiring.