Rallis was appointed to the post of Minister for Public Order in the caretaker cabinet of Panagiotis Kanellopoulos on 3 April 1967. It was in this position that the coup d'état of the Colonels found him on 21 April 1967. Rallis managed to evade capture by the putschists and go to the command centre of the Greek Gendarmerie, from where by radio he tried in vain to get in contact with the III Army Corps and order it to descend onto Athens and suppress the coup. Following the establishment of the Junta of the Colonels, he was arrested thrice, imprisoned and sent to internal exile to the island of Kasos. Among his anti-regime activities were his campaigning against the Junta-sponsored Republic referendum of 1973, and his criticism of the regime through his editorship of the magazine "Politika Themata".