Rabun was a member of both houses of the Georgia General Assembly. In 1805, he was elected to the Georgia House of Representatives, and in 1810, he was elected to the Georgia senate, serving until 1817 as president of the senate. When Governor David B. Mitchell resigned to accept President James Madison's appointment as U.S. agent to the Creek Nation, Rabun became governor of Georgia on March 4, 1817. Mitchell replaced Benjamin Hawkins, who had recently died. Rabun was elected to a full term as governor in November 1817 with the Democratic-Republican Party.