Irving was born in Palo Alto, California. She was the daughter of film and stage director Jules Irving (né Jules Israel) and actress Priscilla Pointer. Irving's brother is a writer and director David Irving, and her sister Katie Irving is a singer and teacher of deaf children. Irving is of Jewish from her father's side and has a little bit of Welsh and Cherokee ancestry on her mother's. She was raised in the Christian Science faith, which means that she believed that God made the earth.