Peter Ball, 84, was jailed for 32 months in October 2015 after admitting a string of historical sex offences against 18 teenagers and young men.
The former bishop of Lewes and Gloucester was released from jail on Friday after serving 16 months.
Phil Johnson said he had served "less than a month for each of the victims".
Ball was sentenced to 32 months for misconduct in public office and 15 months for indecent assaults, to run concurrently, after using "religion as a cloak" to carry out the abuse between the 1970s and 1990s.
Richard Scorer, a lawyer representing a number of Ball's victims, said his early release was "an affront to justice" and "a huge blow to his victims".
"This was a man whose appalling crimes represented a gross and systematic abuse of trust spanning decades," he said.
Mr Johnson, from Eastbourne, who was not one of the 18 people Ball admitted abusing, alleges that Ball inappropriately touched him as a 13-year-old boy.
He said the sentence handed down to him was "in no way proportionate to the crimes committed", and it seemed he had been freed "at the earliest opportunity".
A Church of England spokeswoman said Ball's offences were "a matter of deep shame and regret".
In February 2016, the Archbishop of Canterbury, Most Rev Justin Welby, commissioned an independent review of the Ball case.
Mr Johnson said its publication was not likely "for several more months".
"I think it's utterly ridiculous that it's taken longer to write a report on what happened than it has for Peter Ball to serve his jail sentence," he said.
A Ministry of Justice spokesman said sex offenders were "robustly risk assessed and subject to a strict set of conditions".
"If they fail to comply, they can be recalled to prison," he added.
A man allegedly abused as a child by a former bishop has criticised his early release from jail as "a poor reflection on the criminal justice system".