Skarsgård was married to My Skarsgård from 1975 until they divorced in 2007. They had six children: Alexander (b.1976), Gustaf (b.1980), Sam (b.1982), Bill (b.1990), Eija (b.1992) and Valter (b.1995).