Sir Charles was chosen as the tenth Governor of the colony of New South Wales by Lord Stanley in 1845. FitzRoy replaced Sir George Gipps as governor who had been a strong ruler but had provoked the animosity of many in the colony. It is likely that FitzRoy was chosen because he tended to be more appeasing in his approach. FitzRoy, his wife and his son George arrived in the colony on board on 2 August 1846. Soon after his arrival he was asked to use his influence to procure the disallowance of an act of the Tasmanian legislature imposing a duty of 15% on products imported from New South Wales.