Asch was born in Warsaw, the capital of Poland, to a Jewish family that considered themselves Polish. He grew up in a small town called Łowicz. In 1920, Asch and his family moved to the United States. They lived on the Lower East Side of New York. He did not speak English well, so he found school difficult. He learned English by reading Charles Dickens. Asch got his degree from the College of the City of New York in 1928. In 1930, he married Florence Miller.