On 29 April 1789 he was caught stealing food. Two weeks later he escaped into the bush, taking food, an iron cooking pot, and a musket (gun) taken from a sea army officer (Marine) named Abraham Hand. He could not catch enough animals to eat, and he began to steal food from the settlement. On 26 May he stole food from a brick making gang at Brickfield Hill and was nearly caught. On June 6 he tried to steal food from Zachariah Clark, who was the assistant in charge of the new colony's food supplies. Caesar was caught by a convict named William Saltmarsh.