The Team6 Campaign set about raising £90,000 to build a 6ft 2in (1.87m) bronze statue in Portmeirion, where the cult 1960s TV show was filmed.
But the group said "logistical matters and funding considerations" meant the project could not be achieved in the short term.
Anyone who has donated money to the project will be refunded.
The programme starred McGoohan as Number Six, who is held captive in a mysterious village where the residents are known only by number.
It began shooting in 1966, with all 17 episodes shot in Portmeirion's Italianate village.
McGoohan died in 2009 after almost 50 years on the stage and screen, in a career which saw him win an Emmy and a Bafta TV award.
A Team6 spokesman said the group was "very grateful for the strong early support" but could not achieve the plan "in the window of time originally envisaged".
Plans to build a life-size statue of The Prisoner star Patrick McGoohan in Gwynedd have been shelved.