In 1804 the home was moved to Utzenstorf, a village in the Bernese Emmental. Bitzius gew up there and received his early education. He also had to help his father to cultivate his glebe. In 1812 he went to complete his education at Bern, and in 1820 was received as a pastor. In 1821 he visited the University of Göttingen, but returned home in 1822 to act as his father's assistant. When his father died in 1824, he went to Herzogenbuchsee as a priest. In 1829, he went to Bern. Early in 1831 he went to help the old pastor of the village of Lützelflüh, in the Upper Emmenthal (between Langnau and Burgdorf). In 1832 he was chosen as his successor. He married one of his granddaughters in 1833. He spent the rest of his life there, leaving three children (the son was a pastor, the two daughters married pastors).