Between 1950 and 1955 the club went up and down between Allsvenskan and "Division 2". In 1955 they fell down to Division 2 and stayed there for 20 years; during this time players like Hungarian Ference Marko represented the club at the same time as he lived at the refugee camp in Borgholm in 1957, in 1959 Sune "Mona-Lisa" Andersson, who won an Olympic medal in the 1948 Summer Olympics as a player, as manager, during this time Sten-Åke Johansson played for the club and became the top goalscorer for all time for the club, on 19 October 1975 Kalmar FF defeated Västra Frölunda and returned to Allsvenskan.