In 1898, Herbert became the principal conductor of the Pittsburgh Symphony, a position he held until 1904. Under his leadership, the orchestra became a major American ensemble and was favorably compared by music critics with ensembles like the New York Philharmonic and Boston Symphony Orchestra. The orchestra toured to several major cities during Herbert's years as conductor, notably premiering Herbert's "Auditorium Festival March" for the celebration of the twelfth anniversary of Chicago's Auditorium Theatre in 1901. After a disagreement with the management of the Pittsburgh Symphony in 1904, Herbert resigned, founding the Victor Herbert Orchestra. He conducted their programs of light orchestral music paired with more serious repertoire (as he had done earlier with Anton Seidl's Brighton Beach orchestra concerts) at summer resorts and on tours for most of his remaining years. His orchestra made many acoustical recordings for both Edison Records, from 1909 to 1911, and the Victor Talking Machine Company, from 1911 to 1923. Herbert was also a cello soloist in several Victor recordings as well.