Rabbi Povarsky was born in the city of Kletsk in the Second Polish Republic; his father was Rabbi Dovid Povarsky. When he was three months old, he and his family moved to the city of Baranavichy, which was in the same country; this was because his father was becoming a teacher in the yeshiva of Rabbi Elchonon Wasserman. When he was ten years old, his entire family moved to Mandatory Palestine (present-day Israel), where he attended several schools before joining the Ponovezh Yeshiva, the school where his father was appointed rosh yeshiva. At that point, Baruch Dov was still a student. In 1954, after he got married, he became a teacher in the yeshiva, and later he became the rosh yeshiva.