The 4th U.S.V.I. was organized as a three-year regiment at Point Lookout on October 31, 1864, although only six companies could be induced to enlist. The quality of recruit was not as high as that of the earlier 1st U.S.V.I. The regiment waited at Portsmouth, Virginia, in hopes that more troops could be raised, but calls for men from the west led to its transfer to the frontier at the end of April 1865. 10% deserted before the regiment reached Sioux City, Iowa, in Department of the Northwest, on May 28. Many were arrested and held in confinement before returning to the ranks. Two companies were stationed at Fort Rice, then at Fort Sully; one at Fort Berthold; and three at Fort Randall until June 1866, when the six companies were recalled to Leavenworth, mustering out as they arrived between June 18 and July 2.