Harris was born on November 28, 1950 in Inglewood, New Jersey. He was raised in Tenafly, New Jersey. Harris studied at Tenafly High School and at Columbia University. He married Amy Madigan in 1983. They have one daughter, Lily Dolores, born in 1993. Harris now lives in Los Angeles, California. His father was actor Bob L. Harris. 