At the same time, he founded Congregation Bais Torah in Suffern, New York, and served as its rabbi for the next 24 years. Wein also founded Yeshiva Shaarei Torah of Rockland with a large high school and a smaller post-high school division in 1977. The yeshiva subsequently moved onto the grounds of his synagogue and he served as Rosh Yeshiva (dean) until his move to Israel in 1997. His son, Rabbi Chaim Tzvi Wein, succeeded him as Rosh Yeshiva (along with Mordechai Wolmark, author of Mishnas Mordechai).