On 4 April 1940, Robertson joined the Second AIF himself, with the rank of brigadier, and was allocated the AIF service number VX20321. He was appointed to command the 19th Infantry Brigade, which was then being formed from units made surplus by the reduction of the 6th Division from 12 infantry battalions to nine. All three of its battalions, the 2/4th, 2/8th and 2/11th Infantry Battalions, were initially commanded by over-age officers, but the commander of the 2/4th was replaced by Ivan Dougherty in August. Initially, Dougherty received a cool reception from Robertson, who was disappointed at being unable to select his own battalion commanders, but Dougherty soon made such a good impression that when Robertson went on leave in October 1940 he recommended that Dougherty act as brigade commander, despite the fact that he was the youngest and most junior of Robertson's battalion chiefs.