Root was born in Clinton, New York in 1845. His father Oren Root was a mathematics professor at Hamilton College. Elihu studied at Hamilton College and graduated in 1864. He taught physical education classes in Rome, New York for a couple years before going to law school at New York University in New York City.