Graham Ovenden, from Bodmin Moor, Cornwall, abused children he produced paintings of, in the 1970s and 1980s.
District judge Elizabeth Roscoe later ordered destruction of paintings and photos she judged to be indecent.
Ovenden was jailed in 2013 after his suspended sentence was found by the Court of Appeal to be "unduly lenient".
Judge Roscoe accepted the decision would "invite the wrath of the art world".
And speaking in the Lords, independent peer Lord Clancarty said this "unprecedented judgement has disturbing implications" and it had "huge implications" for the "freedom of expression of many contemporary artists".
Following Ovenden's conviction, the Tate removed more than 30 prints from its online collection.
Labour's Shadow Culture spokesman Lord Stevenson said: "Is the government really saying that it's alright to criticise totalitarian regimes elsewhere for destroying art, but we're not prepared to take action within ourselves?"
Internet Safety Minister Lady Shields said the government cannot interfere with the decision of the court, but said the artist can appeal against the judgement.
She said: "There were works of art in this collection that were relating to the individual specifically that were child sexual abuse images, and you would agree that those should definitely be destroyed."
Ovenden was convicted at Truro Crown Court of six counts of indecency with a child and one of indecent assault.
At his trial, a jury cleared him of three counts of indecency and two of indecent assault.
A decision to destroy works of art by an internationally-renowned artist who abused children who sat for him has been criticised in the House of Lords.