After serving in the military, he returned to playing minor league baseball in 1946. The New York Giants drafted Wilhem in the 1947 minor league draft. This is when he started playing in the major leagues.