Most of modern NY 12E was originally designated as part of NY 3 in 1924. NY 3 was moved onto its current alignment east of Watertown as part of the 1930 renumbering of state highways in New York, at which time its former routing between Watertown and Clayton became NY 12E. From Watertown to Limerick, what is now NY 12E was originally NY 12F. The alignments of the two routes were flipped in the late 1930s. In 1980, Jefferson County assumed maintenance of NY 12E between the Watertown city line and Brownville as part of a highway maintenance swap between the county and the state. This section is now county-maintained as County Route 190 (CR 190).