Forsyth was married to Penny Calvert from 1953 until their divorce in 1973, with whom he had three daughters named Debbie, Julie, and Laura. In 1973, he married Anthea Redfern, the hostess on "The Generation Game". They had two daughters, Charlotte and Louisa, before divorcing in 1979. Asked to judge the 1980 Miss World competition, Forsyth met Puerto Rican beauty queen Wilnelia Merced, who was a fellow judge. They were married from 1983 until his death in 2017. They had one son together, Jonathan Joseph, better known as "JJ". By his six children, Forsyth had nine grandchildren and three great-grandchildren.