De Graaf semi-retired from active politics and became active in the public sector and occupied numerous seats as a nonprofit director on several supervisory boards (Centre for Parliamentary History, Consumers' League and the Anne Vondeling prize) and served on several state commissions and councils on behalf of the government (Public Pension Funds APB, De Koning Commission, Nationaal Comité 4 en 5 mei, Netherlands Film Fund and the Advisory Council for Spatial Planning). De Graaf also worked as a sport administrator for the Royal Dutch Football Association. De Graaf also served as a professor of Ethics for the Royal Marechaussee at the Royal Military Academy from July 2005 until September 2010. In December 2006 De Graaf was nominated as Mayor of Nijmegen, taking office on 8 January 2007. In January 2012 De Graaf was nominated as Chairman of the Executive Board of the Universities of Applied Sciences association, he resigned as Mayor the same day he was installed Chairman from serving from 1 February 2012 until 1 November 2018. De Graaf was elected as a Member of the Senate after the Senate election of 2011, taking office on 7 June 2011 serving as a frontbencher chairing the parliamentary committee for Kingdom Relations and spokesperson for the Interior, Kingdom Relations, European Affairs, Defence and Immigration and Asylum Affairs. After the Senate election of 2015 De Graaf was selected as Parliamentary leader of the Democrats 66 in the Senate, taking office on 9 June 2015. In June 2018 De Graaf was nominated as Vice-President of the Council of State, he resigned as Parliamentary leader on 26 June 2018 and as a Member of the Senate on 20 September 2018 and was installed as Vice-President of the Council of State, taking office on 1 November 2018.