In 1956, they were relegated and moved to their current home, The Bypass Ground. Romsey continued in Division 3 but the early 1960s saw a further decline in fortune, eventually resulting in relegation from the league in 1964. Because the football pyramid was not yet in existence, the club were forced to drop right down to replace their reserves in Junior Division 1 of the Southampton League and it was not until 1968 that they won promotion back to the senior section.