Wilhelm's first assignment in the Giants organization was in Class B with the 1948 Knoxville Smokies, for whom he registered 13 wins and 9 losses. He also spent a few games that season with the Class A Jacksonville Tars of the South Atlantic League, and returned to Jacksonville in 1949, earning a 17–12 win-loss record and a 2.66 earned run average (ERA). Wilhelm was promoted to the Class AAA Minneapolis Millers in 1950, where he was the starting pitcher in 25 of the 35 games he pitched in, registering a 15–11 record with a 4.95 ERA. His role in 1951 with the Millers was the same as the year before, primarily as a starter, but also making eleven relief appearances. His ERA came down to 3.94 in 1951, but his record fell to 11–14.