He first met his wife Vivien, an air stewardess with BOAC, in Bermuda whilst touring there with Yorkshire in 1964, when she was engaged to someone else. He pursued her relentlessly, even though initially she considered him not to be her type. Brian gambled with his love life too: on New Year's Day 1965 he told her that if she didn't agree to marry him, he would never see her again. They married the following March. They went on to have one daughter, Lynn, and a son, Lance.