The Phelps Stokes/Morgan house was designated by the New York City Landmarks Preservation Commission (LPC) in 1965 as one of the first structures to be protected under New York City's Landmarks Law. Next door, the LPC designated the exterior of the library's main building as a city landmark in 1966, and that structure was declared a National Historic Landmark the same year. However, the Lutheran Church had hoped to erect an office structure on the site of the Phelps Stokes/Morgan house, and heavily opposed the house's designation. As a result, in 1974, the landmark status was removed from 231 Madison Avenue, pursuant to a New York Court of Appeals ruling. The Pierpont Morgan Library constructed a five-story, addition to the annex in 1975, to plans by Platt, Wyckoff & Coles; the addition was intended to house storage vaults and offices. In 1982, the main library building's interior was designated a city landmark.