Shortly after becoming a free agent, Foster signed for another Conference Premier side in the form of Southport on 18 June 2014. He was made club captain ahead of the 2014–15 season. Foster made his debut for Southport in a 1–0 home defeat to Forest Green Rovers on the first day of the season. He scored his first goal for the club in just his fourth appearance, his stoppage-time header gave Southport their first win of the season in a 2–1 victory over Altrincham at Haig Avenue on 23 August 2014. Foster made 41 appearances during the campaign in all competitions and scored four times. He was named the club's Player of the Year for the season. He remained at Southport for the 2015–16 campaign and was ever-present during the first half of the campaign. However, Foster started to struggle with the commute to Southport from his home in Barnsley each day and subsequently joined local National League North club Harrogate Town on 24 February 2016, on a contract until the remainder of the season. He made just four appearances during his time at Harrogate, spending the majority of his time there as an unused substitute, and left when his contract expired in April 2016.