From 1954 to 1955, he was part of the Mercedes-Benz factory team, as a junior driver behind Juan Manuel Fangio, Karl Kling, Hermann Lang and later Stirling Moss. When the Silver Arrows came back for the 1954 French Grand Prix to score a 1–2 win, Herrmann drove the fastest lap but had to retire. A podium finish at the 1954 Swiss Grand Prix was his best result in that year as he had to use older versions of the Mercedes-Benz W196, or the least reliable car.