Wanamaker was born on July 11, 1838, in a then-rural, unincorporated area that would in time come to be known as the Grays Ferry neighborhood of South Philadelphia. His parents were John Nelson Wanamaker, a brickmaker and native of Kingwood, New Jersey, and Elizabeth Deshong Kochersperger, daughter of a farmer and innkeeper at Gray's Ferry. Her ancestors came from Rittershoffen in Alsace, France, and from Canton of Bern in Switzerland. At the age of 19 he was hired by the Philadelphia YMCA, he was the first corresponding secretary in the YMCA movement.