Also part of the library grounds is 231 Madison Avenue, an Italianate brownstone house on the southeast corner of Madison Avenue and East 37th Street, which was the home of Isaac Newton Phelps and later J. P. "Jack" Morgan Jr. The house is set behind a barricade composed of a wrought-iron fence atop a brownstone ledge. The house was originally three stories tall and faced with pink stone, but after R. H. Robertson's renovation of 1888, became four stories tall with a raised basement. An office annex to the east, built in 1957, was originally faced with brick. Before the Morgan acquired it in 1988, it was a headquarters of the Lutheran Church.