When Frazier was 15 years old, he had been working on a farm for a family named Bellamy. They were both white men: Mac was younger and more easy-going, and Jim was a little rougher and somewhat backward. One day, a little black boy about 12 years old accidentally damaged one of the Bellamys' tractors. Jim became so enraged he took off his belt and whipped the boy with his belt right there in the field. Frazier saw the event and went back to the packing house on the farm and told his black friends what he had seen. Soon, Jim saw Frazier and asked him why he told what he had witnessed. Joe then told Bellamy he did not know what he was talking about, but Joe did not believe Frazier and told him to get off the farm before he took off his belt again. Frazier told him to keep his pants up because he was not going to use his belt on him. Jim then analyzed Frazier for a bit and eventually said, "Go on, get the hell outta here." Joe knew from that moment it was time for him to leave Beaufort, and he could see only hard times and low rent for himself. Even his Momma could see it. She told Frazier, "Son, if you can't get along with the white folks, then leave home because I don't want anything to happen to you."