Klein was born in Ratno, Ukraine, but in 1910 (at age one) he immigrated with his family to Montreal, Quebec, the city in which he would live most of his life. Ratno had seen a series of pogroms and, like many Ukrainian Jews, Klein's parents sought a safer life elsewhere. As a result of the influx of Jewish immigrants to Montreal, its Jewish community flourished, even though many families lived close to the poverty line. The family of Irving Layton was another notable addition to this community. Klein's father, a devout Orthodox Jew, influenced Klein's early development. The son's early education and literary interests owed much to his plan to become a rabbi when he grew up, a plan that he never fulfilled.