On April 6, 2004, his wife Joan F. Kessler, a long-time member of the Board of Governors of the State Bar of Wisconsin, unseated incumbent Court of Appeals Judge Charles B. Schudson. In November 2004, Fred Kessler was once more elected to the Assembly, from the 12th District (northwestern Milwaukee, a part of Wauwatosa and one precinct in Waukesha County), by a vote of 18,720 to 1761 for Constitution Party nominee Joan Tatarsky. He was assigned to the committees on campaigns and elections; criminal justice and homeland security; the judiciary; and state-federal relations. He was re-elected without opposition in the primary or general election in 2006 and 2008; and in 2010 won re-election with 13,758 votes to 4868 for Republican Sam Hagedorn.