Bolitho had a number of romances. One of the first was Viola Raymond, the wife of an English diplomat. She died while Bolitho and a small number of his crew were stranded in a boat in the tropics, but it was her courage and sacrifice that rallied the crew. Bolitho married twice. His first wife, Cheney Seton, died in a carriage accident whilst carrying their unborn child. His relationship with the second, Belinda, the mother of his child Elizabeth, deteriorated when it became clear that she was nothing like the person he thought he married and was a very selfish individual. Estranged from her and his daughter, he carried on an increasingly public affair with Lady Catherine Somervell, who was his wife in all but name until his death.