"Chocolate Factory" received generally positive reviews from music critics. At Metacritic, which assigns a normalized rating out of 100 to reviews from mainstream critics, the album received an average score of 65, based on 10 reviews, which indicates "generally favorable reviews". Kelefa Sanneh of "The New York Times" found the album "elegant and strange". "Mojo" stated, ""Chocolate Factory" seems positively inspired... An impressively varied opus", and BBC Online stated "doesn't sound like a man with the weight of the world on his shoulders, but a revived soul in spiritual euphoria." Sterling Clover of "The Village Voice" noted the album's "quantity and consistency of sonic presence" and praised Kelly's performance with respect to his songs' structures, stating "Kelly has sussed how to ground himself in tight rhythmic bass and a solid backup chorus, refiguring them into layered and discontinuous sheets of sound". "Rolling Stone"s Anthony DeCurtis cited "Chocolate Factory" as "among the best work of his career" and went on to write "... as a singer, songwriter and producer, he's at the top of his game."