Carrick fell to the bottom of the table on Tuesday when they lost at home to Portadown, and Warrenpoint Town beat Ballymena United.
Haveron's men are one point behind Town and trail second-from-bottom Ballinamallard United by two points.
Carrick play Dungannon away on Saturday and finish at home to Ballinamallard.
Rangers, promoted as Championship One winners last season, were hampered this term by a string of postponements because of playing surface problems at their home ground.
"We are not going to give up - it is not in our nature to throw the towel in," said Haveron.
"We have not been doing enough, but now we need to win our last two games to put it back in our hands.
"The amount of points we have thrown away from winning positions has been hard to take."
The club which finishes bottom of the Premiership will be automatically relegated while the team one from bottom will have a two-legged play-off against the Championship One runners-up.
Warrenpoint, who seemed doomed for the drop after losing 13 games in a row earlier in the season, have recovered to give themselves a fighting chance of survival.
Barry Gray's men have taken seven points from their last five matches, in contrast to Carrick's one.
"We need to continue to show the hunger we have to stay in this league," said Town boss Gray.
At Seaview, Crusaders will be presented with the Premiership trophy before their final home match having retained the Gibson Cup for the first time.
Stephen Baxter's men sealed another league crown by winning 3-1 away to Cliftonville who had themselves won back-to-back titles in 2013 and 2014.
Coleraine, with just one point to show from the last six fixtures, will provide the opposition for the Crues who will again be without suspended defender Josh Robinson.
Runners-up Linfield complete their home programme against Cliftonville while Glenavon, on course to finish third for the second successive year, are away to Glentoran.
Gary Haveron says his Carrick Rangers side need to win their last two matches to stand a chance of avoiding relegation from the Irish Premiership.