McCarthy's side were beaten 1-0 thanks to Leon Best's goal, their fifth defeat of the season at Portman Road.
"The performance was worse than the result, if that can be possible.
"That was as poor as we've played in a long, long time. If we play like that we've no chance of getting into the play-offs," he told BBC Radio Suffolk.
"I'll take a huge portion of the blame, I left the team as it was from Tuesday, maybe we didn't just have the legs - not making excuses we played badly."
McCarthy's side have now lost six of their last 11 league matches, while victory lifted Rotherham out of the Championship relegation zone.
He continued: "The lads always give everything, but I can't accept that performance. I suppose I have to give Rotherham some credit, having scored the goal they defended really well.
"They came with a gameplan, we conceded an awful goal from a short free-kick - there's nothing much more that could have gone wrong in our performance, and that's me included.
Following the defeat, last season's play-off semi-finalists are now four points outside the top six ahead of their next game against 12th-placed Wolves, one of McCarthy's former clubs, on 2 April.
"Each and every one of us has to up our performance," he said.
Ipswich Town manager Mick McCarthy says he will take a "huge portion" of the blame for a "dreadful" display in the home defeat by strugglers Rotherham.