The station was built by the Pennsylvania Railroad and is one of several stations in the U.S. with this name. Until the beginning of the 20th century, there were no trains that went directly into New York City from the west, because of the Hudson River in the way. People had to get off trains in Jersey City, New Jersey and take a ferry to cross the river and reach New York City.