Brooks was born in Durham, North Carolina. He studied Duke University, graduating in 1953 with a Bachelor of Science degree in Physics, and he received a Ph.D. in Applied Mathematics (Computer Science) from Harvard University in 1956, supervised by Howard Aiken.