Ferrer had three siblings. His elder sister, Dr. María Irené Ferrer (July 30, 1915 – November 12, 2004), was a cardiologist and educator, who helped refine the cardiac catheter and electrocardiogram. She died in 2004 in Manhattan, New York City, New York, at age 89 of pneumonia and congestive heart failure. Their brother, Dr. Jose M. Ferrer (November 23, 1912 – December 24, 1982), was a surgeon; he died at age 70 from complications of abdominal surgery. Their younger sister, Teresa Ferrer (March 30, 1919 – February 12, 2002), was the religion editor of "The New York Herald Tribune" and an education editor for "Newsweek". She died at age 82 from a thoracic aneurysm.