Byrne and Sheritt became close friends and found themselves in trouble with the law by falling foul of a local, corrupt police constable. Byrne made his first appearance in court in 1871 on the charge of illegally using a horse, and had to pay a fine of 20 shillings to avoid going to jail. Byrne and Sherritt were later convicted of stealing a bullock and served six months in HM Prison Beechworth. During this imprisonment, Byrne and Sherritt met Jim Kelly who was the brother of Ned Kelly and Dan Kelly. Joe Byrne met Ned in 1876 and the pair soon became firm friends.