Castle was born to William E. Castle and his wife in Cambridge, Massachusetts. His father was a professor of zoology at Harvard, a pioneer in mammalian genetics, and a member of the U.S. National Academy of Sciences. The young Castle was educated in local schools and entered Harvard College in 1914. At the end of his third year of college, he enrolled in Harvard Medical School. Upon graduating from medical school, he did a medical internship at the Massachusetts General Hospital from 1921-1923.