(CNN) -- While several high-profile actors and directors have rallied around Roman Polanski, not everyone in Hollywood believes he should be freed and forgiven.





The case surrounding the arrest of director Roman Polanski has both supporters and critics.





In the face of a  petition signed by notables such as Woody Allen, Martin Scorsese and John Landis calling for the immediate release of Polanski following his detainment in Switzerland, a backlash is building against such support.





"I can't believe that Hollywood has separated itself so completely from American morality," said Paul Petersen, a former child actor and president of A Minor Consideration, which advocates on behalf of young performers. "It is yet another case of Hollywood being out of sync with most of America."





Polanski pleaded guilty in 1977 to having unlawful sexual intercourse with a minor, but fled the United States before he could be sentenced.





In the years since, the director known for "Chinatown" and "Rosemary's Baby" settled in France and avoided authorities until he was arrested Saturday while on his way to the Zurich Film Festival.





In the aftermath of Polanski's arrest last weekend, a number of performers -- including Penelope Cruz, Tilda Swinton and Monica Bellucci -- appear as supporters on a petition which states, "Roman Polanski is a French citizen, a renown and international artist now facing extradition. This extradition, if it takes place, will be heavy in consequences and will take away his freedom."





Polanski's detention has stirred debate over whether he should be returned to the United States to face possible imprisonment.





Supporters make note of his age, the years that have passed, his talent and the victim's own assertion that pursuing the case opens fresh wounds for her as reasons the director should be left alone.





But critics observe that Polanski's supporters might be downplaying his crime.





"I don't think a lot of people take the sexual assault of children very seriously at all, and it's something we battle everyday," said Grier Weeks, executive director of Protect, a national organization to protect children.





"We say at Protect that we are the most bipartisan group in America because we've got everyone from Hollywood liberals like Alison Arngrim [Nellie Oleson] from 'Little House on the Prairie' to extremely conservative people. ... But I will say with the notable exception of a few people from Hollywood, we get blank stares from some of that crowd."





Weeks pointed to the example of a discussion on ABC's "The View," in which panelist and actress Whoopi Goldberg made the comment that she didn't believe the original charges against Polanski were for "rape-rape," as what he views as a pervasive attitude.





"I assume [Goldberg] regrets saying what she said, but she really didn't say anything that is not under the surface anyway," Weeks said. "This is just the attitude, that children are property and crimes against them are not real crimes."  Watch a discussion of about Polanski's supporters »





Goldberg's fellow panelist, Sherri Shepherd, said during the show that she believed Polanski should still be punished.





On Twitter, she tweeted, "Whew... long day at 'The View'... two shows today... hot debate over the Statutory Rapist Polanski. 45-year-old man plies a 13-yr-old w/drugs & Liquor and anally & orally penetrates her w/o her consent is a RAPIST. We hunt down 75-year-old Nazis. We must protect our children."





Actress Kirstie Alley wrote on Twitter "JUST FOR THE RECORD....RAPE IS RAPE...this is one HOLLYWOOD STAR who does not CELEBRATE or DEFEND Roman Polanski..his ART did not RAPE her."





"The Biggest Loser" trainer Jillian Michaels said "If she's [the victim] OK and feels that they are resolved then who are we to say" referring to the now adult victim, Samantha Geimer.





Despite that, Michaels said, "If somebody drugged and raped my 13-year-old I'd shoot them."  Watch stars discuss Polanski's arrest
What did a former child actor say? What is a petition with high-profile names calling for? Who speaks out against Roman Polanski's crime? What has circulated calling for his release?
"I can't believe that Hollywood has separated itself so completely from American morality," the immediate release of Polanski following his detainment in Switzerland, Woody Allen, Martin Scorsese and John Landis petition signed by notables such as Woody Allen, Martin Scorsese and John Landis
question: What did a former child actor say?, answer: "I can't believe that Hollywood has separated itself so completely from American morality," | question: What is a petition with high-profile names calling for?, answer: the immediate release of Polanski following his detainment in Switzerland, | question: Who speaks out against Roman Polanski's crime?, answer: Woody Allen, Martin Scorsese and John Landis | question: What has circulated calling for his release?, answer: petition signed by notables such as Woody Allen, Martin Scorsese and John Landis