Lee attended Eastern View, a school in Fauquier County, Virginia. He may have attended schools in Shirley, Virginia, and in Alexandria, Virginia. His mother instructed him in the Episcopalian faith. Lee attended the United States Military Academy at West Point, and graduated second in the class of 1829.