Ziehrer studied music at the Vienna Conservatory, and began his career as a conductor in 1863. He was also a bandmaster. He was one of the musical rivals of Johann Strauss II, another composer of light music in Vienna.