Schulberg was married four times. In 1936, he married his first wife, actress Virginia "Jigee" Lee Ray. They had one daughter, Victoria, before divorcing in 1942. In 1943, he married Victoria "Vickee" Anderson. They divorced in 1964. They had two children: Stephen (born 1944) and David (born 1946). David was a Vietnam veteran who predeceased his father. In 1964, he married actress Geraldine Brooks. They were married until her death in 1977; they had no children. In 1977, he married Betsy Ann Langman, stepdaughter of real estate developer Robert E. Simon, granddaughter of investment banker Maurice Wertheim and great-granddaughter of US ambassador Henry Morgenthau Sr.; they had two children: Benn and Jessica.