Sweet was born in Highland Park, New Jersey. He studied fine arts at Syracuse University and graduated in 1956. Sweet was famous for making the covers of the fantasy books "The Wheel of Time". He also illustrated covers for the popular "Xanth" series by Piers Anthony, the "Saga of Recluce" by L. E. Modesitt, Jr. and the "Runelords" series by David Farland. Sweet painted the original cover art for Stephen R. Donaldson's series "The Chronicles of Thomas Covenant the Unbeliever".