After Hitler's rise to power in January 1933 and the subsequent suppression of the KPD, Rau became a Central Committee's party instructor for southwest Germany and was active in building an underground party organisation there. On 23 May 1933 Rau was arrested and on 11 December 1934 convicted, together with Bernhard Bästlein, for "preparations to commit high treason" by the People's Court of Germany. He was sentenced to two years imprisonment. After his release from custody, he emigrated to the USSR in August 1935, via Czechoslovakia, and became a deputy chairman of the International Agrarian Institute in Moscow.