Greenhalge attended the public schools of Lowell, where he excelled academically and participated in debating societies. In 1859, he enrolled in Harvard College, where he was a member of the Institute of 1770 and was well regarded as a debater. He left Harvard after three years because his father died, the family finances having suffered a setback due mill closures caused by the American Civil War. He briefly taught school in Chelmsford, Massachusetts and studied law. In 1863, he sought to enlist in the Union Army, but was rejected on account of poor health. He instead secured a civilian job work as a commissary for the army at New Bern, North Carolina. There he contracted malaria, and was sent home in April 1864. He resumed his study of the law, was admitted to the bar in Lowell in 1865.