Wilhem started playing Minor League Baseball in 1942. He served in the United States Army during World War II. Wilhem fought in the Battle of the Bulge and was wounded. He also earned the Purple Heart award. In the military, Wilhem reached the rank of staff sergeant. He was nicknamed "Old Sarge" because he served in the military.