As a teenager, Dudley did not pursue a career in football due to his size, standing only 5 ft 4in at the age of 15, and rarely played the game growing up. However, during his wartime service with the Royal Air Force, he entered a significant growth spurt and grew a further six inches and attended an amateur trial with Southend United in September 1945, eventually turning professional with the side the following month. He made his debut for the club in the final fixtures of the 1945–46 wartime season, scoring five times, and also appeared as a guest player for Colchester United after being stationed in the town during his war service, scoring twice in his only appearance during a 5–0 win over Barry on 9 February 1946.