He remained on the Army rolls during his service in the White House, which limited the ability of Congress to oversee or influence his activities. This circumstance became an issue when he was accused of crimes while in office. In addition to his position in the White House, Grant appointed Babcock Superintendent of Public Buildings and Grounds for Washington, DC. In 1869, Grant sent him on a mission to explore the possibility of annexing the island nation of Santo Domingo to the United States, but the Senate, led by Charles Sumner, rejected the proposal.