Swansea Bay Regional Equality Council said eight cases were reported in the area in June, up from one last year.
However, South Wales Police figures for the entire region showed a slight drop from 113 in June 2015 to 104 in June this year.
Chief executive Taha Idris said the classes were intended to help people in difficult situations.
Self-defence instructor Mohammed Akbor, who is holding classes for both men and women, said his family had experienced abuse.
"We feel that over the last few years, and also after Brexit, racial abuse has increased towards the Muslim community," he said.
"Some members of my family, they have had some racial abuse at them recently, verbal abuse.
"It was quite hurtful... telling them to go back to their country. My children were there at the time and they were quite upset."
Mr Idris said the regional equality council believed a lot of cases go unreported because people are reluctant to contact the police.
"People are having eggs thrown at their windows, dog excrement being left outside their front door. That sort of thing happens quite a lot," he said.
He stressed the self defence classes were intended to help avoid situations rather than making matters worse.
"People were actually coming to us to say in terms of confidence, in terms of going out, just taking part in normal civic duties, people were feeling a lot more inhibited," he said.
"Because of the way they dress, or the way they speak, they are becoming subject to harassment.
"This will enable people to go out and enjoy life as they should be able to do."
Self-defence classes have been launched in Swansea for Muslims and other ethnic groups worried by race hate crimes.