De Graaf was elected as a Member of the House of Representatives after election of 1994, taking office on 17 May 1994 serving as a frontbencher and spokesperson for the Interior, Kingdom Relations, and Law enforcement and deputy spokesperson for Foreign Affairs and European Affairs. After the Parliamentary leader of the Democrats 66 in the House of Representatives Gerrit Jan Wolffensperger announced that he was stepping down as Parliamentary leader in the House of Representatives following increasing criticism on his leadership, the Democrats 66 leadership approached De Graaf as his successor, De Graaf accepted and became the Parliamentary leader in the House of Representatives, taking office on 21 November 1997. After the election of 1998 the new Leader of the Democrats 66 Els Borst was elected as a Member of the House of Representatives and became the Parliamentary leader, taking office on 19 May 1998. Following the cabinet formation of 1998 Borst opted to remain Minister of Health, Welfare and Sport in the Cabinet Kok II and unexpectedly announced that she was stepping down as Leader. De Graaf announced his candidacy to succeed her. De Graaf won the leadership election defeating fellow frontbencher Roger van Boxtel and was elected as Leader and Parliamentary leader, taking office on 30 May 1998. For the election of 2002 De Graaf served as the "Lijsttrekker" (top candidate). The Democrats 66 suffered a big loss, losing 7 seats and fell back as the seventh largest party and now had 7 seats in the House of Representatives. For the election of 2003 De Graaf served for a second time as "Lijsttrekker". The Democrats 66 suffered another loss, losing 1 seat and now had 6 seats in the House of Representatives. On 22 January 2003 De Graaf announced he was stepping down as Leader and Parliamentary leader taking responsibility for the defeat but continued to serve in the House of Representatives as a frontbencher chairing the parliamentary committee for Kingdom Relations. Following the cabinet formation of 2003 De Graaf was appointed as Deputy Prime Minister and Minister for Governmental Reform and Kingdom Relations in the Cabinet Balkenende II, taking office on 27 May 2003. On 23 March 2005 De Graaf resigned after a proposed constitutional reform on elected-mayors was rejected by the Senate.