On 4 March 1957, Barassi married Nancy Kellett, who he had met at work four years earlier. They settled in the eastern suburbs of Melbourne, at Heathmont, and had three children: Susan (born 29 July 1960), Ron (born 23 June 1962) and Richard (born 13 February 1964). The couple separated in 1975 and Barassi married Cherryl Copeland in 1981. During his coaching career at North Melbourne he survived a car crash, which caused life-threatening injuries and resulted in the loss of his spleen. His passenger, former St Kilda player and Brownlow medallist Neil Roberts, was also hurt. Barassi used a motorised buggy and a wheelchair for a short time. Despite this setback, he attended training nights at Arden Street and could be seen directing players with assistants.