He married Remigia Ferrari-Scacco in 1947. They had two children. They divorced in 1979. He married Ann Fleming in 1979. Their marriage would last until Brooke's death in 2015. They had one child. He wrote a book about his life, "Bridging the Divide: My Life", in 2007. He lived in Coral Gables, Florida.