Media playback is not supported on this device
Fox-Pitt incurred time penalties to score 30.4 aboard Chilli Morning, having led after Sunday's dressage.
The 47-year-old will go into Tuesday's final show-jumping phase in 21st place, with Great Britain lying seventh in the team standings.
"It's obviously very disappointing," Fox-Pitt told BBC Sport.
"To be up there and cock it up is a blow. I hope he [Chilli Morning] is OK, life goes on. So far, Team GB are not doing what we dreamed of."
Fox-Pitt is competing just 10 months after being induced into a coma following a fall.
As he suffered disappointment on Monday, fellow Britons Pippa Funnell (28th), Kitty King (34th) and Gemma Tattersall (44th) also picked up penalties during their rounds to leave Britain struggling to challenge for a team medal.
The damage was done for Fox-Pitt when Chilli Morning missed an obstacle, with Fox-Pitt admitting: "I hadn't really felt that was a tight line for him so it was my fault, because he's such a genuine horse.
"If he could have seen it, he would have jumped it."
Christopher Burton set the pace with a brilliant round on Santano II to lead the individual standings and take Australia top of the team competition, ahead of New Zealand, France and Germany.
Fox-Pitt, Britain's most successful rider, suffered a head trauma in a cross-country fall last October and only returned to action in April.
He said it was "special" just to be competing in Rio, let alone taking an early lead on Sunday.
A spokesperson for Poole Hospital, where Fox-Pitt underwent rehabilitation after his injuries, told BBC Sport: "All at Poole Hospital are thrilled with the progress William has made, and have been watching his return not only to health but to the highest levels of sporting achievement with enormous interest and pride.
"William's recovery has been remarkable and we wish him the best of luck in his Olympic endeavours."
Britain's William Fox-Pitt slipped from first to well outside the medal places after the cross-country phase of the Olympic eventing competition in Rio.