The NY 96 designation was originally assigned to present-day NY 2 from NY 7 in Troy to Route 2 at the Massachusetts state line. In 1942, the alignments of NY 2 and NY 96 were swapped, placing NY 96 on the Owego–Rochester routing. The only major change to NY 96 since that time came in the early 1950s, when the route was realigned between Candor and Ithaca to follow a new routing via Spencer. The Candor–Spencer portion of the alignment had been part of NY 53 during the 1920s and part of NY 223 since 1930. NY 223 was truncated to its current eastern terminus at NY 224 near Van Etten as part of NY 96's realignment. Between Spencer and Ithaca, NY 96 overlapped with NY 34, which had occupied that segment of highway since the 1930 renumbering.