In 1953, Haynes left the Globetrotters after an acrimonious split with Abe Saperstein, the team's owner. After quitting the team, he turned down a $35,000 a year offer from the Philadelphia Warriors that would have made him the second-highest paid player in the NBA to found his own barnstorming team, the Harlem Magicians, after finding out that Saperstein was a part-owner of the Warriors. He also received an offer to play for the Minneapolis Lakers in 1955, but he turned down that opportunity as well. Despite a series of legal battles with Saperstein, Haynes attended his funeral in 1966, although he later joked he was only there to "make sure he didn't jump out of the casket".