Rubin was born on August 29, 1938, in New York City to Jewish parents Sylvia (née Seiderman) and Alexander Rubin. He moved to Miami Beach, Florida, at an early age and graduated from Miami Beach High School. In 1960, Rubin graduated with an A.B. "summa cum laude" in economics from Harvard College. He then attended Harvard Law School for three days before leaving to see the world. He later attended the London School of Economics and received an LL.B. from Yale Law School in 1964.