Cox was born to Howard Ellis Cox and Anne Crane Delafield (Finch) Cox in Southampton Hospital in Southampton, New York. He attended Westhampton Beach Elementary School and Allen-Stevenson School in New York City. Cox is named for his grandfather, Judge Edward R. Finch, a prominent New York jurist who served as a Justice of the New York State Supreme Court (1915–43), Presiding Justice of the New York Supreme Court Appellate Division, First Department, and Associate Judge on the New York Court of Appeals. His father, Howard Ellis Cox, was a decorated World War II aviator, New York lawyer, and Long Island real estate developer.