The film of the incident at a Johannesburg branch of the Spur chain of restaurants has been widely shared.
In it, the two can be heard arguing and hurling insults at each other, while a group of children are sitting at the table watching the row unfold.
The video appears to have touched a nerve in a country where racism is still a sore point.
It is not clear what started the altercation but both adults say that their child was bullied by the other's child.
The confrontation escalated when the man threatened to hit the woman, who accused him of bullying her because she was black.
"You can't come here and bully meâ€¦ this is a democratic South Africa in case you had not noticed," the incensed woman says.
Although the man does not say anything specifically racist, many have interpreted his treatment of the woman as being based on her skin colour.
At one point he lifts up and shakes the table at which the woman's family was seated.
A number of people including customers are seen trying to intervene until he eventually leaves.
People have taken to social media to express their dismay at how the restaurant "failed to protect" the woman.
The restaurant chain, whose managers have been criticised for not stepping in, has apologised for the drama at one of its busiest stores.
"The man in the video is not welcome at any Spur restaurant nationally.
We do not condone any forms of violence against women or children, irrespective of the circumstances," the company said in statement shared on social media.
The incident is believed to have happened at the weekend but the video has only just surfaced.
A video of a white man threatening a black woman inside a popular restaurant has caused outrage in South Africa.