Right-back Naughton (hamstring) and left-back Olsson (ankle) both missed the 2-0 defeat at Bournemouth.
Naughton was also ruled out of the loss at Hull, but both are fit to face Boro at the Liberty Stadium.
"We are in good shape going into this game," said Swansea boss Paul Clement.
After reverses at Hull and Bournemouth, the Swans are 17th in the table, five points ahead of Middlesbrough, who are one from bottom of the table.
Clement, who confirmed all his players had returned from international duty fit, says the returning defenders have been missed as he had been forced to reshuffle his resources.
"I think so because of the way we have played going into those games," the former Bayern Munich assistant manager added.
"They have been very influential both defending and particularly attacking from those positions, so we are very happy to have them back."
Clement says the pressure to avoid relegation has to be embraced by his players, but realises the importance of the game against Middlesbrough.
"Nine games to go and the fact that we did not pick up any points in the last two it's important to get back points on the board again," he said.
"We are going to use our home form  momentum to do our very best to get points against Middlesbrough.
"If we win, I think it would put them in a difficult situation. It would not be over for them but would make things more difficult, of course."
Asked whether he had set a points target for his side, Clement said: "What's the highest amount of points a team has gone down with? 42? So we have to get more than that."
Swansea City will welcome back defenders Martin Olsson and Kyle Naughton for Sunday's Premier League match against relegation rivals Middlesbrough.