I am 65 years old and of mixed racial heritage. Even on the reservation we called criminals thugs. No blacks lived on the reservation back then. 
I now live in Alaska for 44 years, we call bums who steal, beat up people, and make threats to little old ladies THUGS. THUGS come in all colors, shapes, and sizes. 
Thugs are bullies, they are usually not very smart and depend on their muscles not brains to get them through life. 
Ruffians, hooligans, and vandals were names that my grandparents called the neighborhood THUGS.  Villains and gangsters were 70's words for THUGS.
These days people throw around "RACIST" so much that they take away from what a real racist is. A THUG can be a racist and a RACIST can be a THUG. 
Look at it this way, in the US we call a bathroom by many terms, latrines, toilet, head, powder room, bathroom, restroom, crapper, out-house. In England and other former British colonies, they call them water closets, loos, John's, lavatory, the Ladies and the Gentleman's.