Kerr was born on November 15, 1931 in New York City, New York. His parents were British actor Geoffrey Kerr and American actress June Walker. He studied at Phillips Exeter Academy and at Harvard University.