Carter retired in 1910. In 1919, he changed his surname to "Gilbert-Carter" and in the early 1920s, he moved back to Barbados, and lived at Ilaro Court, which had been designed and built by his wife. 