The Bluebirds were second from bottom when former boss Paul Trollope was sacked in October.
Under current manager Neil Warnock, Cardiff have climbed up to 13th place with three games left in the campaign.
"We were right down there at the start of the season so I think we've come a long way," said Peltier.
"If we finish in the top half, it just makes you think what could have been if we'd started off properly. We would have been right up there."
Asked if he was envious of the clubs competing for promotion, Peltier said: "Yeah, definitely. You're always looking to see the results and who's in and who's not in. It's going right down to the wire at both ends."
Peltier returned from an ankle injury to make his first appearance since January as Cardiff beat Nottingham Forest on Easter Monday.
The 30-year-old, who played for 64 minutes against Forest before being substituted, had initially feared his season was over.
"The plan was to get through it and get it sorted at the end of the season, but against Preston [in January] my ankle just locked up, and I thought 'enough is enough'," said Peltier.
"I went to see the surgeon and he operated on it straight away. It was quite a big operation so I was glad to get it sorted. I feel good and I'm just looking to get myself back fit.
"I was due to be out for the season, but I've come back quite quick. The gaffer's been telling me not to push it, but I've been wanting to come back in the squad. If I'd have left it, it would have been a long time from when I'd been injured until pre-season, so I didn't want to be out that long."
With little to play for between now and the end of the season, Warnock has repeatedly said in recent weeks he is already planning for the next campaign.
Despite being described by his manager as one of the best right-backs in the Championship, Peltier is not taking his place in the squad for granted.
"I wouldn't say fine - you never know in football do you? Everyone's fighting for places and I think competition in the team is very good," he added.
"It will just make us stronger, there's a good team spirit in the dressing room and there's no jealousy or anything like that. It will be good to get some fresh faces in and see what happens."
Cardiff City defender Lee Peltier says their strong second half to the season shows they can compete with the leading teams in the Championship.