Shortly after the beginning of Operation Barbarossa, Shepilov joined the Soviet "People's Militia" (Narodnoe Opolcheniye) in July 1941 and was a Political commissar of its Moscow component during the Battle of Moscow in 1941–1942. In 1942–1943 he was the political commissar of the 23rd Guards Army and in 1944–1946 of the 4th Guards Army, ending the war with the rank of Major General. Between May 1945 and February 1946, Shepilov was one of the top Soviet officials in Vienna during the early stages of the Soviet occupation of eastern parts of Austria.