Media playback is not supported on this device
Lowestoft's Ogogo had to settle for a bronze after losing 16-9 to Brazil's Esquiva Falcao Florentino.
However, it was his fourth fight of the competition, one more than his seeded opponent, who had a first-round bye.
"I feel I'm better than him, but he was sharper than me on the day," the 23-year-old told BBC Radio Suffolk.
After a five-month spell out of the ring recovering from shoulder surgery, Ogogo had hoped to better the silver medal he won at the 2010 Commonwealth Games by winning an Olympic gold.
And he boxed superbly to beat world number one Ievgen Khytrov of Ukraine in his second fight and followed that with a 15-10 win over Germany's Stefan Hartel to reach the semi-finals.
"The Ukrainian hadn't been beaten in a long time and he was really physically strong," Ogogo said. "He punched hard and knocks you about for fun.
"I fought tooth and nail to beat him - it was really hard - while he [Florentino] boxed a lesser opponent.
Media playback is not supported on this device
"Then I boxed the German and he was very big and strong and that fight took a lot out of me again. He [Florentino] boxed a Hungarian who wasn't that brilliant.
"He had two fights against lesser quality than I did and I definitely think it was a case of one fight too much when I fought him."
There was to be no golden finish for Florentino, who was outpointed 14-13 by Japan's Ryota Murata in the middleweight final.
"I feel immensely proud," Ogogo added. "Obviously the medal I got isn't the one I wanted and always thought about having, but I worked hard and I feel immensely proud just being part of this team."
Great Britain won three boxing gold medals at London 2012, through super-heavyweight Anthony Joshua, bantamweight Luke Campbell and Nicola Adams in the women's competition.
Middleweight Anthony Ogogo is convinced he could have reached the Olympic final at London 2012 if his path to the last four had not been so tough.