Adolf Gustav Thorsen was born to a family of craftsmen, just outside Halse og Harkmark, a former municipality in Mandal. His parents were Elesæus Thorsen (1835–1886), a cabinetmaker and Anne Aanensdatter (1835–1907). He had three brothers, of whom Emanuel Vigeland (originally Thorsen) became a noted artist. As a youth, he was sent to Oslo where he learned wood carving at a local school. However, the sudden death of his father compelled him to move back to Mandal to help his family. Gustav lived for a time with his grandparents on a farm called Mjunebrokka in Vigeland, an old farm in Valle parish, Lindesnes municipality in Vest-Agder county.