In 1864, Flanders campaigned for governor and finished in third place behind Michael Hahn and Fellows. He was appointed by Republicans as the first Supervising Special Agent of the Freedmen’s Bureau, Department of the Gulf. At the same time, he led the movement to create a local Republican Party in Louisiana. He formed the 'Friends of Universal Suffrage' with other Louisiana Unionists (known as scalawags to opponents), as well as free men of color (who had been free before the war) and freedmen; they were working to gain black suffrage and to repeal the Louisiana Black Codes. These laws had been passed to control the movement of freedmen. Fearful of the black majority in many Louisiana districts, most white Democrats opposed giving freedmen suffrage, especially after Confederate veterans were temporarily disenfranchised unless they took a loyalty oath. The tension over the rights of freed slaves escalated into New Orleans riot of 1866, in which whites attacked blacks.