The segment of modern NY 120 between Westchester Avenue and the north end of the overlap with NY 22 was originally designated as part of Route 1, an unsigned legislative route, by the New York State Legislature in 1908. Route 1 approached Purchase Street from the west on Westchester Avenue and continued north from Armonk on what is now NY 22. NY 120, meanwhile, was assigned as part of the 1930 renumbering of state highways in New York to the portion of its modern alignment north of the junction of Purchase Street and Westchester Avenue (then NY 119) in Harrison. It was extended south to US 1 in Rye by way of Purchase Street and Highland Road.