Stewart was appointed Minister of State for Prisons with responsibility for prisons and probation in England and Wales in January 2018. He was appointed in the aftermath of a highly critical leaked report on the state of HMP Liverpool, in which the inspector described it as the "worst prison he had ever seen" with piles of rubbish, rats, soaring violence and drug use and poor health provision. Stewart immediately visited the prison and, testifying before the Justice Select Committee, announced his determination to clean up prisons in England and Wales. This advocacy of a "back to basics" approach was recorded in "The Guardian", with Stewart writing an opinion piece in the publication, entitled "I strongly believe we can improve our prisons and make progress".