Born in Kroz, Lithuania, Joseph had studied in the Volozhin yeshiva under Naftali Zvi Yehuda Berlin; he was known there as Rav Yaakov Charif ("Rabbi Jacob Sharp") because of his sharp mind. He was one of the main disciples of Yisroel Salanter, and in 1883 had been appointed the "maggid" (preacher) of Vilna. Beth Hamedrash Hagodol, the Eldridge Street Synagogue, and 13 other Lower East Side synagogues had raised $2,500 (today $) towards the creation of a European style "kehilla" to oversee New York's Orthodox community, and had imported Joseph in an attempt to achieve that (ultimately unfulfilled) goal. Joseph's salary was to be the then-substantial $2,500 per year, "with an additional $1000 for rent, furnishings, and utilities". Though Joseph's appointment was, in part, intended to bring prestige to the downtown Orthodox congregations, his primary task as Chief Rabbi was to bring order and regulation to New York's chaotic kosher slaughtering industry.